Merge branch 'feat/ibm-memory' of https://github.com/joaomdmoura/crewAI into feat/ibm-memory

This commit replaces .netrc with uv environment variables for installing
tools from private repositories. To store credentials, I created a new
and reusable settings file for the CLI in
`$HOME/.config/crewai/settings.json`.

The issue with .netrc files is that they are applied system-wide and are
scoped by hostname, meaning we can't differentiate tool repositories
requests from regular requests to CrewAI's API.

docs/concepts/llms.mdx

@@ -25,52 +25,55 @@ By default, CrewAI uses the `gpt-4o-mini` model. It uses environment variables i
 - `OPENAI_API_BASE`
 - `OPENAI_API_KEY`
 
-### 2. String Identifier
-
-```python Code
-agent = Agent(llm="gpt-4o", ...)
-```
-
-### 3. LLM Instance
-
-List of [more providers](https://docs.litellm.ai/docs/providers).
-
-```python Code
-from crewai import LLM
-
-llm = LLM(model="gpt-4", temperature=0.7)
-agent = Agent(llm=llm, ...)
-```
-
-### 4. Custom LLM Objects
+### 2. Custom LLM Objects
 
 Pass a custom LLM implementation or object from another library.
 
+See below for examples.
+
+<Tabs>
+    <Tab title="String Identifier">
+    ```python Code
+    agent = Agent(llm="gpt-4o", ...)
+    ```
+    </Tab>
+
+    <Tab title="LLM Instance">
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(model="gpt-4", temperature=0.7)
+    agent = Agent(llm=llm, ...)
+    ```
+    </Tab>
+</Tabs>
+
 ## Connecting to OpenAI-Compatible LLMs
 
 You can connect to OpenAI-compatible LLMs using either environment variables or by setting specific attributes on the LLM class:
 
-1. Using environment variables:
+<Tabs>
+    <Tab title="Using Environment Variables">
+    ```python Code
+    import os
 
-```python Code
-import os
+    os.environ["OPENAI_API_KEY"] = "your-api-key"
+    os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
+    ```
+    </Tab>
+    <Tab title="Using LLM Class Attributes">
+    ```python Code
+    from crewai import LLM
 
-os.environ["OPENAI_API_KEY"] = "your-api-key"
-os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
-```
-
-2. Using LLM class attributes:
-
-```python Code
-from crewai import LLM
-
-llm = LLM(
-    model="custom-model-name",
-    api_key="your-api-key",
-    base_url="https://api.your-provider.com/v1"
-)
-agent = Agent(llm=llm, ...)
-```
+    llm = LLM(
+        model="custom-model-name",
+        api_key="your-api-key",
+        base_url="https://api.your-provider.com/v1"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Tab>
+</Tabs>
 
 ## LLM Configuration Options
 
@@ -97,55 +100,149 @@ When configuring an LLM for your agent, you have access to a wide range of param
 | **api_key**          | `str`             | Your API key for authentication.                                                                 |
 
 
-## OpenAI Example Configuration
+These are examples of how to configure LLMs for your agent.
 
-```python Code
-from crewai import LLM
+<AccordionGroup>    
+    <Accordion title="OpenAI">
 
-llm = LLM(
-    model="gpt-4",
-    temperature=0.8,
-    max_tokens=150,
-    top_p=0.9,
-    frequency_penalty=0.1,
-    presence_penalty=0.1,
-    stop=["END"],
-    seed=42,
-    base_url="https://api.openai.com/v1",
-    api_key="your-api-key-here"
-)
-agent = Agent(llm=llm, ...)
-```
+    ```python Code
+    from crewai import LLM
 
-## Cerebras Example Configuration
+    llm = LLM(
+        model="gpt-4",
+        temperature=0.8,
+        max_tokens=150,
+        top_p=0.9,
+        frequency_penalty=0.1,
+        presence_penalty=0.1,
+        stop=["END"],
+        seed=42,
+        base_url="https://api.openai.com/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
 
-```python Code
-from crewai import LLM
+    <Accordion title="Cerebras">
 
-llm = LLM(
-    model="cerebras/llama-3.1-70b",
-    base_url="https://api.cerebras.ai/v1",
-    api_key="your-api-key-here"
-)
-agent = Agent(llm=llm, ...) 
-```
+    ```python Code
+    from crewai import LLM
 
-## Using Ollama (Local LLMs)
+    llm = LLM(
+        model="cerebras/llama-3.1-70b",
+        base_url="https://api.cerebras.ai/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...) 
+    ```
+    </Accordion>
+ 
+    <Accordion title="Ollama (Local LLMs)">
 
-CrewAI supports using Ollama for running open-source models locally:
+    CrewAI supports using Ollama for running open-source models locally:
 
-1. Install Ollama: [ollama.ai](https://ollama.ai/)
-2. Run a model: `ollama run llama2`
-3. Configure agent:
+    1. Install Ollama: [ollama.ai](https://ollama.ai/)
+    2. Run a model: `ollama run llama2`
+    3. Configure agent:
 
-```python Code
-from crewai import LLM
+    ```python Code
+    from crewai import LLM
 
-agent = Agent(
-    llm=LLM(model="ollama/llama3.1", base_url="http://localhost:11434"),
-    ...
-)
-```
+    agent = Agent(
+        llm=LLM(
+            model="ollama/llama3.1", 
+            base_url="http://localhost:11434"
+        ),
+        ...
+    )
+    ```
+    </Accordion>
+
+    <Accordion title="Groq">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="groq/llama3-8b-8192", 
+        base_url="https://api.groq.com/openai/v1", 
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+
+    <Accordion title="Anthropic">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="anthropic/claude-3-5-sonnet-20241022",
+        base_url="https://api.anthropic.com/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+
+    <Accordion title="Fireworks">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="fireworks/meta-llama-3.1-8b-instruct",
+        base_url="https://api.fireworks.ai/inference/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+
+    <Accordion title="Gemini">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="gemini/gemini-1.5-flash",
+        base_url="https://api.gemini.google.com/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+
+    <Accordion title="Perplexity AI (pplx-api)">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="perplexity/mistral-7b-instruct",
+        base_url="https://api.perplexity.ai/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+
+    <Accordion title="IBM watsonx.ai">
+
+    ```python Code
+    from crewai import LLM
+
+    llm = LLM(
+        model="watsonx/ibm/granite-13b-chat-v2",
+        base_url="https://api.watsonx.ai/v1",
+        api_key="your-api-key-here"
+    )
+    agent = Agent(llm=llm, ...)
+    ```
+    </Accordion>
+</AccordionGroup>
 
 ## Changing the Base API URL
 

docs/concepts/memory.mdx

@@ -254,6 +254,31 @@ my_crew = Crew(
 )
 ```
 
+### Using Watson embeddings
+
+```python Code
+from crewai import Crew, Agent, Task, Process
+
+# Note: Ensure you have installed and imported `ibm_watsonx_ai` for Watson embeddings to work.
+
+my_crew = Crew(
+    agents=[...],
+    tasks=[...],
+    process=Process.sequential,
+    memory=True,
+    verbose=True,
+    embedder={
+        "provider": "watson",
+        "config": {
+            "model": "<model_name>",
+            "api_url": "<api_url>",
+            "api_key": "<YOUR_API_KEY>",
+            "project_id": "<YOUR_PROJECT_ID>",
+        }
+    }
+)
+```
+
 ### Resetting Memory
 
 ```shell

src/crewai/agent.py

@@ -8,7 +8,6 @@ from pydantic import Field, InstanceOf, PrivateAttr, model_validator
 from crewai.agents import CacheHandler
 from crewai.agents.agent_builder.base_agent import BaseAgent
 from crewai.agents.crew_agent_executor import CrewAgentExecutor
-from crewai.cli.constants import ENV_VARS
 from crewai.llm import LLM
 from crewai.memory.contextual.contextual_memory import ContextualMemory
 from crewai.tools.agent_tools import AgentTools
@@ -131,12 +130,8 @@ class Agent(BaseAgent):
             # If it's already an LLM instance, keep it as is
             pass
         elif self.llm is None:
-            # Determine the model name from environment variables or use default
-            model_name = (
-                os.environ.get("OPENAI_MODEL_NAME")
-                or os.environ.get("MODEL")
-                or "gpt-4o-mini"
-            )
+            # If it's None, use environment variables or default
+            model_name = os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
             llm_params = {"model": model_name}
 
             api_base = os.environ.get("OPENAI_API_BASE") or os.environ.get(
@@ -145,39 +140,9 @@ class Agent(BaseAgent):
             if api_base:
                 llm_params["base_url"] = api_base
 
-            # Iterate over all environment variables to find matching API keys or use defaults
-            for provider, env_vars in ENV_VARS.items():
-                for env_var in env_vars:
-                    # Check if the environment variable is set
-                    if "key_name" in env_var:
-                        env_value = os.environ.get(env_var["key_name"])
-                        if env_value:
-                            # Map key names containing "API_KEY" to "api_key"
-                            key_name = (
-                                "api_key"
-                                if "API_KEY" in env_var["key_name"]
-                                else env_var["key_name"]
-                            )
-                            # Map key names containing "API_BASE" to "api_base"
-                            key_name = (
-                                "api_base"
-                                if "API_BASE" in env_var["key_name"]
-                                else key_name
-                            )
-                            # Map key names containing "API_VERSION" to "api_version"
-                            key_name = (
-                                "api_version"
-                                if "API_VERSION" in env_var["key_name"]
-                                else key_name
-                            )
-                            llm_params[key_name] = env_value
-                    # Check for default values if the environment variable is not set
-                    elif env_var.get("default", False):
-                        for key, value in env_var.items():
-                            if key not in ["prompt", "key_name", "default"]:
-                                # Only add default if the key is already set in os.environ
-                                if key in os.environ:
-                                    llm_params[key] = value
+            api_key = os.environ.get("OPENAI_API_KEY")
+            if api_key:
+                llm_params["api_key"] = api_key
 
             self.llm = LLM(**llm_params)
         else:

src/crewai/cli/config.py

@@ -0,0 +1,38 @@
+import json
+from pathlib import Path
+from pydantic import BaseModel, Field
+from typing import Optional
+
+DEFAULT_CONFIG_PATH = Path.home() / ".config" / "crewai" / "settings.json"
+
+class Settings(BaseModel):
+    tool_repository_username: Optional[str] = Field(None, description="Username for interacting with the Tool Repository")
+    tool_repository_password: Optional[str] = Field(None, description="Password for interacting with the Tool Repository")
+    config_path: Path = Field(default=DEFAULT_CONFIG_PATH, exclude=True)
+
+    def __init__(self, config_path: Path = DEFAULT_CONFIG_PATH, **data):
+        """Load Settings from config path"""
+        config_path.parent.mkdir(parents=True, exist_ok=True)
+
+        file_data = {}
+        if config_path.is_file():
+            try:
+                with config_path.open("r") as f:
+                    file_data = json.load(f)
+            except json.JSONDecodeError:
+                file_data = {}
+
+        merged_data = {**file_data, **data}
+        super().__init__(config_path=config_path, **merged_data)
+
+    def dump(self) -> None:
+        """Save current settings to settings.json"""
+        if self.config_path.is_file():
+            with self.config_path.open("r") as f:
+                existing_data = json.load(f)
+        else:
+            existing_data = {}
+
+        updated_data = {**existing_data, **self.model_dump(exclude_unset=True)}
+        with self.config_path.open("w") as f:
+            json.dump(updated_data, f, indent=4)

src/crewai/cli/constants.py

@@ -1,168 +1,19 @@
 ENV_VARS = {
-    "openai": [
-        {
-            "prompt": "Enter your OPENAI API key (press Enter to skip)",
-            "key_name": "OPENAI_API_KEY",
-        }
-    ],
-    "anthropic": [
-        {
-            "prompt": "Enter your ANTHROPIC API key (press Enter to skip)",
-            "key_name": "ANTHROPIC_API_KEY",
-        }
-    ],
-    "gemini": [
-        {
-            "prompt": "Enter your GEMINI API key (press Enter to skip)",
-            "key_name": "GEMINI_API_KEY",
-        }
-    ],
-    "groq": [
-        {
-            "prompt": "Enter your GROQ API key (press Enter to skip)",
-            "key_name": "GROQ_API_KEY",
-        }
-    ],
-    "watson": [
-        {
-            "prompt": "Enter your WATSONX URL (press Enter to skip)",
-            "key_name": "WATSONX_URL",
-        },
-        {
-            "prompt": "Enter your WATSONX API Key (press Enter to skip)",
-            "key_name": "WATSONX_APIKEY",
-        },
-        {
-            "prompt": "Enter your WATSONX Project Id (press Enter to skip)",
-            "key_name": "WATSONX_PROJECT_ID",
-        },
-    ],
-    "ollama": [
-        {
-            "default": True,
-            "API_BASE": "http://localhost:11434",
-        }
-    ],
-    "bedrock": [
-        {
-            "prompt": "Enter your AWS Access Key ID (press Enter to skip)",
-            "key_name": "AWS_ACCESS_KEY_ID",
-        },
-        {
-            "prompt": "Enter your AWS Secret Access Key (press Enter to skip)",
-            "key_name": "AWS_SECRET_ACCESS_KEY",
-        },
-        {
-            "prompt": "Enter your AWS Region Name (press Enter to skip)",
-            "key_name": "AWS_REGION_NAME",
-        },
-    ],
-    "azure": [
-        {
-            "prompt": "Enter your Azure deployment name (must start with 'azure/')",
-            "key_name": "model",
-        },
-        {
-            "prompt": "Enter your AZURE API key (press Enter to skip)",
-            "key_name": "AZURE_API_KEY",
-        },
-        {
-            "prompt": "Enter your AZURE API base URL (press Enter to skip)",
-            "key_name": "AZURE_API_BASE",
-        },
-        {
-            "prompt": "Enter your AZURE API version (press Enter to skip)",
-            "key_name": "AZURE_API_VERSION",
-        },
-    ],
-    "cerebras": [
-        {
-            "prompt": "Enter your Cerebras model name (must start with 'cerebras/')",
-            "key_name": "model",
-        },
-        {
-            "prompt": "Enter your Cerebras API version (press Enter to skip)",
-            "key_name": "CEREBRAS_API_KEY",
-        },
-    ],
+    'openai': ['OPENAI_API_KEY'],
+    'anthropic': ['ANTHROPIC_API_KEY'],
+    'gemini': ['GEMINI_API_KEY'],
+    'groq': ['GROQ_API_KEY'],
+    'ollama': ['FAKE_KEY'],
 }
 
-
-PROVIDERS = [
-    "openai",
-    "anthropic",
-    "gemini",
-    "groq",
-    "ollama",
-    "watson",
-    "bedrock",
-    "azure",
-    "cerebras",
-]
+PROVIDERS = ['openai', 'anthropic', 'gemini', 'groq', 'ollama']
 
 MODELS = {
-    "openai": ["gpt-4", "gpt-4o", "gpt-4o-mini", "o1-mini", "o1-preview"],
-    "anthropic": [
-        "claude-3-5-sonnet-20240620",
-        "claude-3-sonnet-20240229",
-        "claude-3-opus-20240229",
-        "claude-3-haiku-20240307",
-    ],
-    "gemini": [
-        "gemini/gemini-1.5-flash",
-        "gemini/gemini-1.5-pro",
-        "gemini/gemini-gemma-2-9b-it",
-        "gemini/gemini-gemma-2-27b-it",
-    ],
-    "groq": [
-        "groq/llama-3.1-8b-instant",
-        "groq/llama-3.1-70b-versatile",
-        "groq/llama-3.1-405b-reasoning",
-        "groq/gemma2-9b-it",
-        "groq/gemma-7b-it",
-    ],
-    "ollama": ["ollama/llama3.1", "ollama/mixtral"],
-    "watson": [
-        "watsonx/google/flan-t5-xxl",
-        "watsonx/google/flan-ul2",
-        "watsonx/bigscience/mt0-xxl",
-        "watsonx/eleutherai/gpt-neox-20b",
-        "watsonx/ibm/mpt-7b-instruct2",
-        "watsonx/bigcode/starcoder",
-        "watsonx/meta-llama/llama-2-70b-chat",
-        "watsonx/meta-llama/llama-2-13b-chat",
-        "watsonx/ibm/granite-13b-instruct-v1",
-        "watsonx/ibm/granite-13b-chat-v1",
-        "watsonx/google/flan-t5-xl",
-        "watsonx/ibm/granite-13b-chat-v2",
-        "watsonx/ibm/granite-13b-instruct-v2",
-        "watsonx/elyza/elyza-japanese-llama-2-7b-instruct",
-        "watsonx/ibm-mistralai/mixtral-8x7b-instruct-v01-q",
-    ],
-    "bedrock": [
-        "bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0",
-        "bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
-        "bedrock/anthropic.claude-3-haiku-20240307-v1:0",
-        "bedrock/anthropic.claude-3-opus-20240229-v1:0",
-        "bedrock/anthropic.claude-v2:1",
-        "bedrock/anthropic.claude-v2",
-        "bedrock/anthropic.claude-instant-v1",
-        "bedrock/meta.llama3-1-405b-instruct-v1:0",
-        "bedrock/meta.llama3-1-70b-instruct-v1:0",
-        "bedrock/meta.llama3-1-8b-instruct-v1:0",
-        "bedrock/meta.llama3-70b-instruct-v1:0",
-        "bedrock/meta.llama3-8b-instruct-v1:0",
-        "bedrock/amazon.titan-text-lite-v1",
-        "bedrock/amazon.titan-text-express-v1",
-        "bedrock/cohere.command-text-v14",
-        "bedrock/ai21.j2-mid-v1",
-        "bedrock/ai21.j2-ultra-v1",
-        "bedrock/ai21.jamba-instruct-v1:0",
-        "bedrock/meta.llama2-13b-chat-v1",
-        "bedrock/meta.llama2-70b-chat-v1",
-        "bedrock/mistral.mistral-7b-instruct-v0:2",
-        "bedrock/mistral.mixtral-8x7b-instruct-v0:1",
-    ],
+    'openai': ['gpt-4', 'gpt-4o', 'gpt-4o-mini', 'o1-mini', 'o1-preview'],
+    'anthropic': ['claude-3-5-sonnet-20240620', 'claude-3-sonnet-20240229', 'claude-3-opus-20240229', 'claude-3-haiku-20240307'],
+    'gemini': ['gemini-1.5-flash', 'gemini-1.5-pro', 'gemini-gemma-2-9b-it', 'gemini-gemma-2-27b-it'],
+    'groq': ['llama-3.1-8b-instant', 'llama-3.1-70b-versatile', 'llama-3.1-405b-reasoning', 'gemma2-9b-it', 'gemma-7b-it'],
+    'ollama': ['llama3.1', 'mixtral'],
 }
 
-JSON_URL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
+JSON_URL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

src/crewai/cli/create_crew.py

@@ -1,11 +1,11 @@
-import shutil
 import sys
 from pathlib import Path
 
 import click
 
-from crewai.cli.constants import ENV_VARS, MODELS
+from crewai.cli.constants import ENV_VARS
 from crewai.cli.provider import (
+    PROVIDERS,
     get_provider_data,
     select_model,
     select_provider,
@@ -29,20 +29,20 @@ def create_folder_structure(name, parent_folder=None):
             click.secho("Operation cancelled.", fg="yellow")
             sys.exit(0)
         click.secho(f"Overriding folder {folder_name}...", fg="green", bold=True)
-        shutil.rmtree(folder_path)  # Delete the existing folder and its contents
+    else:
+        click.secho(
+            f"Creating {'crew' if parent_folder else 'folder'} {folder_name}...",
+            fg="green",
+            bold=True,
+        )
 
-    click.secho(
-        f"Creating {'crew' if parent_folder else 'folder'} {folder_name}...",
-        fg="green",
-        bold=True,
-    )
-
-    folder_path.mkdir(parents=True)
-    (folder_path / "tests").mkdir(exist_ok=True)
-    if not parent_folder:
-        (folder_path / "src" / folder_name).mkdir(parents=True)
-        (folder_path / "src" / folder_name / "tools").mkdir(parents=True)
-        (folder_path / "src" / folder_name / "config").mkdir(parents=True)
+    if not folder_path.exists():
+        folder_path.mkdir(parents=True)
+        (folder_path / "tests").mkdir(exist_ok=True)
+        if not parent_folder:
+            (folder_path / "src" / folder_name).mkdir(parents=True)
+            (folder_path / "src" / folder_name / "tools").mkdir(parents=True)
+            (folder_path / "src" / folder_name / "config").mkdir(parents=True)
 
     return folder_path, folder_name, class_name
 
@@ -92,10 +92,7 @@ def create_crew(name, provider=None, skip_provider=False, parent_folder=None):
 
         existing_provider = None
         for provider, env_keys in ENV_VARS.items():
-            if any(
-                "key_name" in details and details["key_name"] in env_vars
-                for details in env_keys
-            ):
+            if any(key in env_vars for key in env_keys):
                 existing_provider = provider
                 break
 
@@ -121,48 +118,47 @@ def create_crew(name, provider=None, skip_provider=False, parent_folder=None):
                 "No provider selected. Please try again or press 'q' to exit.", fg="red"
             )
 
-        # Check if the selected provider has predefined models
-        if selected_provider in MODELS and MODELS[selected_provider]:
-            while True:
-                selected_model = select_model(selected_provider, provider_models)
-                if selected_model is None:  # User typed 'q'
-                    click.secho("Exiting...", fg="yellow")
-                    sys.exit(0)
-                if selected_model:  # Valid selection
-                    break
-                click.secho(
-                    "No model selected. Please try again or press 'q' to exit.",
-                    fg="red",
-                )
-            env_vars["MODEL"] = selected_model
-
-        # Check if the selected provider requires API keys
-        if selected_provider in ENV_VARS:
-            provider_env_vars = ENV_VARS[selected_provider]
-            for details in provider_env_vars:
-                if details.get("default", False):
-                    # Automatically add default key-value pairs
-                    for key, value in details.items():
-                        if key not in ["prompt", "key_name", "default"]:
-                            env_vars[key] = value
-                elif "key_name" in details:
-                    # Prompt for non-default key-value pairs
-                    prompt = details["prompt"]
-                    key_name = details["key_name"]
-                    api_key_value = click.prompt(prompt, default="", show_default=False)
-
-                    if api_key_value.strip():
-                        env_vars[key_name] = api_key_value
-
-        if env_vars:
-            write_env_file(folder_path, env_vars)
-            click.secho("API keys and model saved to .env file", fg="green")
-        else:
+        while True:
+            selected_model = select_model(selected_provider, provider_models)
+            if selected_model is None:  # User typed 'q'
+                click.secho("Exiting...", fg="yellow")
+                sys.exit(0)
+            if selected_model:  # Valid selection
+                break
             click.secho(
-                "No API keys provided. Skipping .env file creation.", fg="yellow"
+                "No model selected. Please try again or press 'q' to exit.", fg="red"
             )
 
-        click.secho(f"Selected model: {env_vars.get('MODEL', 'N/A')}", fg="green")
+        if selected_provider in PROVIDERS:
+            api_key_var = ENV_VARS[selected_provider][0]
+        else:
+            api_key_var = click.prompt(
+                f"Enter the environment variable name for your {selected_provider.capitalize()} API key",
+                type=str,
+                default="",
+            )
+
+        api_key_value = ""
+        click.echo(
+            f"Enter your {selected_provider.capitalize()} API key (press Enter to skip): ",
+            nl=False,
+        )
+        try:
+            api_key_value = input()
+        except (KeyboardInterrupt, EOFError):
+            api_key_value = ""
+
+        if api_key_value.strip():
+            env_vars = {api_key_var: api_key_value}
+            write_env_file(folder_path, env_vars)
+            click.secho("API key saved to .env file", fg="green")
+        else:
+            click.secho(
+                "No API key provided. Skipping .env file creation.", fg="yellow"
+            )
+
+        env_vars["MODEL"] = selected_model
+        click.secho(f"Selected model: {selected_model}", fg="green")
 
     package_dir = Path(__file__).parent
     templates_dir = package_dir / "templates" / "crew"

src/crewai/cli/templates/crew/crew.py

@@ -48,4 +48,4 @@ class {{crew_name}}Crew():
 			process=Process.sequential,
 			verbose=True,
 			# process=Process.hierarchical, # In case you wanna use that instead https://docs.crewai.com/how-to/Hierarchical/
-		)
+		)

src/crewai/cli/templates/crew/main.py

@@ -1,11 +1,7 @@
 #!/usr/bin/env python
 import sys
-import warnings
-
 from {{folder_name}}.crew import {{crew_name}}Crew
 
-warnings.filterwarnings("ignore", category=SyntaxWarning, module="pysbd")
-
 # This main file is intended to be a way for you to run your
 # crew locally, so refrain from adding unnecessary logic into this file.
 # Replace with inputs you want to test with, it will automatically

src/crewai/cli/tools/main.py

@@ -1,17 +1,15 @@
 import base64
 import os
-import platform
 import subprocess
 import tempfile
 from pathlib import Path
-from netrc import netrc
-import stat
 
 import click
 from rich.console import Console
 
 from crewai.cli import git
 from crewai.cli.command import BaseCommand, PlusAPIMixin
+from crewai.cli.config import Settings
 from crewai.cli.utils import (
     get_project_description,
     get_project_name,
@@ -153,26 +151,16 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
             raise SystemExit
 
         login_response_json = login_response.json()
-        self._set_netrc_credentials(login_response_json["credential"])
+
+        settings = Settings()
+        settings.tool_repository_username = login_response_json["credential"]["username"]
+        settings.tool_repository_password = login_response_json["credential"]["password"]
+        settings.dump()
 
         console.print(
             "Successfully authenticated to the tool repository.", style="bold green"
         )
 
-    def _set_netrc_credentials(self, credentials, netrc_path=None):
-        if not netrc_path:
-            netrc_filename = "_netrc" if platform.system() == "Windows" else ".netrc"
-            netrc_path = Path.home() / netrc_filename
-            netrc_path.touch(mode=stat.S_IRUSR | stat.S_IWUSR, exist_ok=True)
-
-        netrc_instance = netrc(file=netrc_path)
-        netrc_instance.hosts["app.crewai.com"] = (credentials["username"], "", credentials["password"])
-
-        with open(netrc_path, 'w') as file:
-            file.write(str(netrc_instance))
-
-        console.print(f"Added credentials to {netrc_path}", style="bold green")
-
     def _add_package(self, tool_details):
         tool_handle = tool_details["handle"]
         repository_handle = tool_details["repository"]["handle"]
@@ -187,7 +175,11 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
             tool_handle,
         ]
         add_package_result = subprocess.run(
-            add_package_command, capture_output=False, text=True, check=True
+            add_package_command,
+            capture_output=False,
+            env=self._build_env_with_credentials(repository_handle),
+            text=True,
+            check=True
         )
 
         if add_package_result.stderr:
@@ -206,3 +198,13 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
                 "[bold yellow]Tip:[/bold yellow] Navigate to a different directory and try again."
             )
             raise SystemExit
+
+    def _build_env_with_credentials(self, repository_handle: str):
+        repository_handle = repository_handle.upper().replace("-", "_")
+        settings = Settings()
+
+        env = os.environ.copy()
+        env[f"UV_INDEX_{repository_handle}_USERNAME"] = str(settings.tool_repository_username or "")
+        env[f"UV_INDEX_{repository_handle}_PASSWORD"] = str(settings.tool_repository_password or "")
+
+        return env

src/crewai/memory/contextual/contextual_memory.py

@@ -34,6 +34,7 @@ class ContextualMemory:
         formatted_results = "\n".join(
             [f"- {result['context']}" for result in stm_results]
         )
+        print("formatted_results stm", formatted_results)
         return f"Recent Insights:\n{formatted_results}" if stm_results else ""
 
     def _fetch_ltm_context(self, task) -> Optional[str]:
@@ -53,6 +54,8 @@ class ContextualMemory:
         formatted_results = list(dict.fromkeys(formatted_results))
         formatted_results = "\n".join([f"- {result}" for result in formatted_results])  # type: ignore # Incompatible types in assignment (expression has type "str", variable has type "list[str]")
 
+        print("formatted_results ltm", formatted_results)
+
         return f"Historical Data:\n{formatted_results}" if ltm_results else ""
 
     def _fetch_entity_context(self, query) -> str:
@@ -64,4 +67,5 @@ class ContextualMemory:
         formatted_results = "\n".join(
             [f"- {result['context']}" for result in em_results]  # type: ignore #  Invalid index type "str" for "str"; expected type "SupportsIndex | slice"
         )
+        print("formatted_results em", formatted_results)
         return f"Entities:\n{formatted_results}" if em_results else ""

src/crewai/memory/storage/kickoff_task_outputs_storage.py

@@ -70,7 +70,7 @@ class KickoffTaskOutputsSQLiteStorage:
                         task.expected_output,
                         json.dumps(output, cls=CrewJSONEncoder),
                         task_index,
-                        json.dumps(inputs),
+                        json.dumps(inputs, cls=CrewJSONEncoder),
                         was_replayed,
                     ),
                 )

src/crewai/memory/storage/rag_storage.py

@@ -4,13 +4,13 @@ import logging
 import os
 import shutil
 import uuid
-from typing import Any, Dict, List, Optional
-from crewai.memory.storage.base_rag_storage import BaseRAGStorage
-from crewai.utilities.paths import db_storage_path
+from typing import Any, Dict, List, Optional, cast
+
+from chromadb import Documents, EmbeddingFunction, Embeddings
 from chromadb.api import ClientAPI
 from chromadb.api.types import validate_embedding_function
-from chromadb import Documents, EmbeddingFunction, Embeddings
-from typing import cast
+from crewai.memory.storage.base_rag_storage import BaseRAGStorage
+from crewai.utilities.paths import db_storage_path
 
 
 @contextlib.contextmanager
@@ -21,9 +21,11 @@ def suppress_logging(
     logger = logging.getLogger(logger_name)
     original_level = logger.getEffectiveLevel()
     logger.setLevel(level)
-    with contextlib.redirect_stdout(io.StringIO()), contextlib.redirect_stderr(
-        io.StringIO()
-    ), contextlib.suppress(UserWarning):
+    with (
+        contextlib.redirect_stdout(io.StringIO()),
+        contextlib.redirect_stderr(io.StringIO()),
+        contextlib.suppress(UserWarning),
+    ):
         yield
     logger.setLevel(original_level)
 
@@ -113,12 +115,52 @@ class RAGStorage(BaseRAGStorage):
                 self.embedder_config = embedding_functions.HuggingFaceEmbeddingServer(
                     url=config.get("api_url"),
                 )
+            elif provider == "watson":
+                try:
+                    import ibm_watsonx_ai.foundation_models as watson_models
+                    from ibm_watsonx_ai import Credentials
+                    from ibm_watsonx_ai.metanames import (
+                        EmbedTextParamsMetaNames as EmbedParams,
+                    )
+                except ImportError as e:
+                    raise ImportError(
+                        "IBM Watson dependencies are not installed. Please install them to use Watson embedding."
+                    ) from e
+
+                class WatsonEmbeddingFunction(EmbeddingFunction):
+                    def __call__(self, input: Documents) -> Embeddings:
+                        if isinstance(input, str):
+                            input = [input]
+
+                        embed_params = {
+                            EmbedParams.TRUNCATE_INPUT_TOKENS: 3,
+                            EmbedParams.RETURN_OPTIONS: {"input_text": True},
+                        }
+
+                        embedding = watson_models.Embeddings(
+                            model_id=config.get("model"),
+                            params=embed_params,
+                            credentials=Credentials(
+                                api_key=config.get("api_key"), url=config.get("api_url")
+                            ),
+                            project_id=config.get("project_id"),
+                        )
+
+                        try:
+                            embeddings = embedding.embed_documents(input)
+                            return cast(Embeddings, embeddings)
+
+                        except Exception as e:
+                            print("Error during Watson embedding:", e)
+                            raise e
+
+                self.embedder_config = WatsonEmbeddingFunction()
             else:
                 raise Exception(
-                    f"Unsupported embedding provider: {provider}, supported providers: [openai, azure, ollama, vertexai, google, cohere, huggingface]"
+                    f"Unsupported embedding provider: {provider}, supported providers: [openai, azure, ollama, vertexai, google, cohere, huggingface, watson]"
                 )
         else:
-            validate_embedding_function(self.embedder_config)  # type: ignore # used for validating embedder_config if defined a embedding function/class
+            validate_embedding_function(self.embedder_config)
             self.embedder_config = self.embedder_config
 
     def _initialize_app(self):

src/crewai/telemetry/telemetry.py

@@ -21,7 +21,7 @@ with suppress_warnings():
 
 
 from opentelemetry import trace  # noqa: E402
-from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter  # noqa: E402
+from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter # noqa: E402
 from opentelemetry.sdk.resources import SERVICE_NAME, Resource  # noqa: E402
 from opentelemetry.sdk.trace import TracerProvider  # noqa: E402
 from opentelemetry.sdk.trace.export import BatchSpanProcessor  # noqa: E402
@@ -48,6 +48,10 @@ class Telemetry:
     def __init__(self):
         self.ready = False
         self.trace_set = False
+
+        if os.getenv("OTEL_SDK_DISABLED", "false").lower() == "true":
+            return
+
         try:
             telemetry_endpoint = "https://telemetry.crewai.com:4319"
             self.resource = Resource(

src/crewai/utilities/crew_json_encoder.py

@@ -2,13 +2,14 @@ from datetime import datetime, date
 import json
 from uuid import UUID
 from pydantic import BaseModel
+from decimal import Decimal
 
 
 class CrewJSONEncoder(json.JSONEncoder):
     def default(self, obj):
         if isinstance(obj, BaseModel):
             return self._handle_pydantic_model(obj)
-        elif isinstance(obj, UUID):
+        elif isinstance(obj, UUID) or isinstance(obj, Decimal):
             return str(obj)
 
         elif isinstance(obj, datetime) or isinstance(obj, date):

tests/cli/config_test.py

@@ -0,0 +1,109 @@
+import unittest
+import json
+import tempfile
+import shutil
+from pathlib import Path
+from crewai.cli.config import Settings
+
+class TestSettings(unittest.TestCase):
+    def setUp(self):
+        self.test_dir = Path(tempfile.mkdtemp())
+        self.config_path = self.test_dir / "settings.json"
+
+    def tearDown(self):
+        shutil.rmtree(self.test_dir)
+
+    def test_empty_initialization(self):
+        settings = Settings(config_path=self.config_path)
+        self.assertIsNone(settings.tool_repository_username)
+        self.assertIsNone(settings.tool_repository_password)
+
+    def test_initialization_with_data(self):
+        settings = Settings(
+            config_path=self.config_path,
+            tool_repository_username="user1"
+        )
+        self.assertEqual(settings.tool_repository_username, "user1")
+        self.assertIsNone(settings.tool_repository_password)
+
+    def test_initialization_with_existing_file(self):
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        with self.config_path.open("w") as f:
+            json.dump({"tool_repository_username": "file_user"}, f)
+
+        settings = Settings(config_path=self.config_path)
+        self.assertEqual(settings.tool_repository_username, "file_user")
+
+    def test_merge_file_and_input_data(self):
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        with self.config_path.open("w") as f:
+            json.dump({
+                "tool_repository_username": "file_user",
+                "tool_repository_password": "file_pass"
+            }, f)
+
+        settings = Settings(
+            config_path=self.config_path,
+            tool_repository_username="new_user"
+        )
+        self.assertEqual(settings.tool_repository_username, "new_user")
+        self.assertEqual(settings.tool_repository_password, "file_pass")
+
+    def test_dump_new_settings(self):
+        settings = Settings(
+            config_path=self.config_path,
+            tool_repository_username="user1"
+        )
+        settings.dump()
+
+        with self.config_path.open("r") as f:
+            saved_data = json.load(f)
+
+        self.assertEqual(saved_data["tool_repository_username"], "user1")
+
+    def test_update_existing_settings(self):
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        with self.config_path.open("w") as f:
+            json.dump({"existing_setting": "value"}, f)
+
+        settings = Settings(
+            config_path=self.config_path,
+            tool_repository_username="user1"
+        )
+        settings.dump()
+
+        with self.config_path.open("r") as f:
+            saved_data = json.load(f)
+
+        self.assertEqual(saved_data["existing_setting"], "value")
+        self.assertEqual(saved_data["tool_repository_username"], "user1")
+
+    def test_none_values(self):
+        settings = Settings(
+            config_path=self.config_path,
+            tool_repository_username=None
+        )
+        settings.dump()
+
+        with self.config_path.open("r") as f:
+            saved_data = json.load(f)
+
+        self.assertIsNone(saved_data.get("tool_repository_username"))
+
+    def test_invalid_json_in_config(self):
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        with self.config_path.open("w") as f:
+            f.write("invalid json")
+
+        try:
+            settings = Settings(config_path=self.config_path)
+            self.assertIsNone(settings.tool_repository_username)
+        except json.JSONDecodeError:
+            self.fail("Settings initialization should handle invalid JSON")
+
+    def test_empty_config_file(self):
+        self.config_path.parent.mkdir(parents=True, exist_ok=True)
+        self.config_path.touch()
+
+        settings = Settings(config_path=self.config_path)
+        self.assertIsNone(settings.tool_repository_username)

tests/cli/tools/test_main.py

@@ -82,6 +82,7 @@ def test_install_success(mock_get, mock_subprocess_run):
         capture_output=False,
         text=True,
         check=True,
+        env=unittest.mock.ANY
     )
 
     assert "Succesfully installed sample-tool" in output