Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vector dimension erroneously being sent to ada-002 model when using Azure OpenAI #6334

Open
1 task done
g-parki opened this issue Nov 14, 2024 · 2 comments
Open
1 task done
Labels

Comments

@g-parki
Copy link

g-parki commented Nov 14, 2024

How to reproduce this bug?

import weaviate
import weaviate.classes.config as wvc

client = weaviate.use_async_with_weaviate_cloud(...)
await client.connect()

collection = await client.collections.create(
    name="MyCollection",
    properties=[
        wvc.Property(name="content", data_type=wvc.DataType.TEXT),
    ],
    vectorizer_config=wvc.Configure.Vectorizer.text2vec_azure_openai(
        resource_name="myCompanyResource",
        deployment_id="myCompany-text-embedding-ada-002",
    ),
)

await collection.data.insert({"content": "foo"})

What is the expected behavior?

Data is uploaded correctly

What is the actual behavior?

UnexpectedStatusCodeError: Object was not added! Unexpected status code: 500, with response body: 
{'error': [{'message': 'update vector: connection to: Azure OpenAI API failed with status:
400 request-id: 0c80d879-09c2-44b1-84ac-2bd4052efb8b
error: This model does not support specifying dimensions.'}]}

Supporting information

I'm seeing this on 1.27.3 and the issue goes away when I downgrade to 1.26.6. I suspect there's an issue with the change in #5987 - will comment here with my findings

Server Version

1.27.3

Weaviate Setup

Single Node

Nodes count

1

Code of Conduct

@g-parki g-parki added the bug label Nov 14, 2024
@g-parki
Copy link
Author

g-parki commented Nov 14, 2024

Here's my theory - sorry if it's a little long-winded. As a starting point, the AzureOpenAI vectorizer config interface is different from the OpenAI vectorizer config interface. The Azure class has no model while the OpenAI class does have model, modelVersion, type_, and dimensions. This is what's in the Python client:

class _Text2VecAzureOpenAIConfig(_VectorizerConfigCreate):
    vectorizer: Union[Vectorizers, _EnumLikeStr] = Field(
        default=Vectorizers.TEXT2VEC_OPENAI, frozen=True, exclude=True
    )
    baseURL: Optional[AnyHttpUrl]
    resourceName: str
    deploymentId: str
    vectorizeClassName: bool

class _Text2VecOpenAIConfig(_VectorizerConfigCreate):
    vectorizer: Union[Vectorizers, _EnumLikeStr] = Field(
        default=Vectorizers.TEXT2VEC_OPENAI, frozen=True, exclude=True
    )
    baseURL: Optional[AnyHttpUrl]
    dimensions: Optional[int]
    model: Optional[str]
    modelVersion: Optional[str]
    type_: Optional[OpenAIType]
    vectorizeClassName: bool

In the vectorizer module, a default model is assigned, which is a v3 model, and doesn't take into account any of the deploymentId information from the Azure interface:

const (
DefaultOpenAIDocumentType = "text"
DefaultOpenAIModel = "text-embedding-3-small"
DefaultVectorizeClassName = true
DefaultPropertyIndexed = true
DefaultVectorizePropertyName = false
DefaultBaseURL = "https://api.openai.com"
DefaultApiVersion = "2024-02-01"
LowerCaseInput = false
)

func (cs *classSettings) Model() string {
return cs.BaseClassSettings.GetPropertyAsString("model", DefaultOpenAIModel)
}

The Dimensions is then determined based on the model:

func PickDefaultDimensions(model string) *int64 {
if model == TextEmbedding3Small {
return &TextEmbedding3SmallDefaultDimensions
}
if model == TextEmbedding3Large {
return &TextEmbedding3LargeDefaultDimensions
}
return nil
}

func (cs *classSettings) Dimensions() *int64 {
defaultValue := PickDefaultDimensions(cs.Model())
return cs.BaseClassSettings.GetPropertyAsInt64("dimensions", defaultValue)
}

And this dimension ultimately winds up getting passed unconditionally in the request body:

func (v *client) getEmbeddingsRequest(input []string, model string, isAzure bool, dimensions *int64) embeddingsRequest {
if isAzure {
return embeddingsRequest{Input: input, Dimensions: dimensions}
}
return embeddingsRequest{Input: input, Model: model, Dimensions: dimensions}
}

@g-parki
Copy link
Author

g-parki commented Nov 15, 2024

I set up a dev weaviate/weaviate-python-client environment and I'm able to work around the issue client-side by expanding _Text2VecAzureOpenAIConfig to include model, modelVersion, type_ and dimensions. The workaround looks like:

    vectorizer_config=wvc.Configure.Vectorizer.text2vec_azure_openai(
        resource_name="myCompanyResource",
        deployment_id="myCompany-text-embedding-ada-002",
        model="ada",
        model_version="002",
        type_="text"
    )

I can submit a PR in the client repos for this change, but from a UX perspective I think the validations on the db should be updated as well, like:

  • If Azure and no model was provided, attempt to use the deploymentId as the model (when deploying a model in Azure, it pre-fills the deployment name with the OpenAI model name, but allows the user to rename it. So I think it's safe to assume there are many deploymentIds which match the model name)
  • If Azure - don't use a default model. Require a user-specified model (or deploymentId matching an accepted model) for everything derived from it, e.g. if dimensions is specified, require that the model was provided by the user

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant