Metadata-Version: 2.1
Name: superlinked
Version: 3.9.0
Summary: The Superlinked vector computing library
License: Apache-2.0
Author: Superlinked Release
Author-email: release@superlinked.com
Requires-Python: >=3.10,<3.11
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Provides-Extra: batch
Provides-Extra: deployment-executor
Provides-Extra: deployment-poller
Provides-Extra: interactive
Provides-Extra: profiler
Requires-Dist: accelerate (==0.27.2) ; extra == "interactive"
Requires-Dist: adlfs (==2023.12.0) ; extra == "deployment-executor"
Requires-Dist: altair (==5.2.0) ; extra == "interactive"
Requires-Dist: beartype (>=0.17.2,<0.18.0)
Requires-Dist: boto3 (==1.34.13) ; extra == "deployment-poller"
Requires-Dist: boto3-stubs[s3] (==1.34.13) ; extra == "deployment-poller"
Requires-Dist: cerberus (==1.3.5) ; extra == "deployment-executor" or extra == "deployment-poller"
Requires-Dist: dask[complete,distributed] (==2024.1.0) ; extra == "deployment-executor"
Requires-Dist: deltalake (==0.15.1) ; extra == "deployment-executor"
Requires-Dist: fastapi (==0.109.0) ; extra == "deployment-executor"
Requires-Dist: fastapi-restful (>=0.5.0,<0.6.0) ; extra == "deployment-executor"
Requires-Dist: fsspec (==2023.12.2) ; extra == "deployment-executor"
Requires-Dist: furl (>=2.1.3,<3.0.0) ; extra == "deployment-executor"
Requires-Dist: gcsfs (==2023.12.2.post1) ; extra == "deployment-executor"
Requires-Dist: google-auth (==2.26.1) ; extra == "deployment-poller"
Requires-Dist: google-cloud-storage (==2.14.0) ; extra == "deployment-poller"
Requires-Dist: graphviz (>=0.20.1,<0.21.0) ; extra == "profiler"
Requires-Dist: httpx (==0.26.0) ; extra == "deployment-executor"
Requires-Dist: inject (>=5.2.0,<6.0.0) ; extra == "deployment-executor"
Requires-Dist: ipywidgets (==8.1.2) ; extra == "interactive"
Requires-Dist: jupyter (>=1.0.0,<2.0.0) ; extra == "interactive" or extra == "profiler"
Requires-Dist: numpy (==1.22.4)
Requires-Dist: pandas (>=2.2.1,<3.0.0)
Requires-Dist: pydantic-settings (>=2.2.1,<3.0.0) ; extra == "deployment-executor"
Requires-Dist: pyspark (==3.5.0) ; extra == "batch"
Requires-Dist: pyyaml (==6.0.1) ; extra == "deployment-poller"
Requires-Dist: requests (==2.27.1) ; extra == "deployment-poller"
Requires-Dist: s3fs (==2023.12.2) ; extra == "deployment-executor"
Requires-Dist: sentence-transformers (==2.2.2)
Requires-Dist: snakeviz (>=2.2.0,<2.3.0) ; extra == "profiler"
Requires-Dist: supervisor (==4.2.5) ; extra == "deployment-executor"
Requires-Dist: transformers (==4.38.2) ; extra == "interactive"
Requires-Dist: typing-inspect (>=0.9.0,<0.10.0) ; extra == "deployment-executor"
Requires-Dist: typing_extensions (>=3.10.0.2) ; extra == "deployment-executor"
Requires-Dist: umap-learn (==0.5.5) ; extra == "interactive"
Requires-Dist: uvicorn (==0.15.0) ; extra == "deployment-executor"
Requires-Dist: yelp-gprof2dot (>=1.2.0,<1.3.0) ; extra == "profiler"
Description-Content-Type: text/markdown

# Superlinked 

Superlinked is a declarative Python SDK that enables you to turn complex data into vectors, in a way that fits the modern data stack and works with your favorite Vector Databases.

3 key areas of focus:

1. Custom embedding model creation that fits your complex data entities.
1. ETL for your vector index for both streaming and batch use-cases.
1. Vector-native query language that helps you convert hybrid search queries to pure vector queries.

Visit [Superlinked](https://superlinked.com/) for more information about the company behind this product and our other initiatives.

## Use-cases

- **RAG**: [HR Knowledgebase](https://github.com/superlinked/superlinked/blob/main/notebook/rag_hr_knowledgebase.ipynb)
- **Semantic Search**: [Movie Recommendations](https://github.com/superlinked/superlinked/blob/main/notebook/semantic_search_netflix_titles.ipynb), [Business News](https://github.com/superlinked/superlinked/blob/main/notebook/semantic_search_news.ipynb)
- **Recommendation Systems**: [E-commerce](https://github.com/superlinked/superlinked/blob/main/notebook/recommendations_e_commerce.ipynb)
- **Analytics**: [User Acquisition](https://github.com/superlinked/superlinked/blob/main/notebook/analytics_user_acquisition.ipynb)

You can check a full list of examples [here](https://github.com/superlinked/superlinked/tree/main/notebook).

## Reference

1. Describe your data using Python classes with the [@schema](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/common/schema/schema.md) decorator.
2. Describe your vector embeddings from building blocks with [Spaces](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/dsl/space/index.md).
3. Combine your embeddings into a queryable [Index](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/dsl/index/index.m.md).
4. Define your search with dynamic parameters and weights as a [Query](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/dsl/query/query.md).
5. Load your data using a [Source](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/dsl/source/index.md).
6. Define your transformations with a [Parser](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/common/parser) (e.g.: from [`pd.DataFrame`](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/common/parser/dataframe_parser.md)). 
7. Run your configuration with an [Executor](https://github.com/superlinked/superlinked/blob/main/docs/superlinked/framework/dsl/executor/in_memory/in_memory_executor.md).

You can check a list of our [features](https://github.com/superlinked/superlinked/tree/main/notebook/feature) or head to our [documentation](https://github.com/superlinked/superlinked/tree/main/docs).
  
## Try it out

Example on how to use Superlinked to experiment with the semantic search use-case. 

### Pre-requisities

#### In a notebook

Install the superlinked library: 
```
%pip install superlinked
```

#### As a script 
Ensure your python version is 3.10.x.

```commandline
$> python -V
Python 3.10.9
```

If your python version is not `3.10.x` you might use [pyenv](https://github.com/pyenv/pyenv) to install it. 

Upgrade pip and install the superlinked library

```commandline
$> python -m pip install --upgrade pip
$> python -m pip install superlinked
```

### Run the example

>First run will take slightly longer as it has to download the embedding model.  

```python
from superlinked.framework.common.schema.schema import schema
from superlinked.framework.common.schema.schema_object import String
from superlinked.framework.common.schema.id_schema_object import IdField
from superlinked.framework.dsl.space.text_similarity_space import TextSimilaritySpace
from superlinked.framework.dsl.index.index import Index
from superlinked.framework.dsl.query.param import Param
from superlinked.framework.dsl.query.query import Query
from superlinked.framework.dsl.source.in_memory_source import InMemorySource
from superlinked.framework.dsl.executor.in_memory.in_memory_executor import InMemoryExecutor


@schema # Desribe your schemas.
class Document:
    id: IdField  # Each schema should have exactly one `IdField`.
    body: String # Use `String` for text fields.

document = Document()

relevance_space = TextSimilaritySpace(text=document.body, model="sentence-transformers/all-mpnet-base-v2") # Select your semantic embedding model.
document_index = Index([relevance_space]) # Combine your spaces to a queryable index.

query = Query(document_index).find(document).similar(relevance_space.text, Param("query_text")) # Define your query with dynamic parameters.

source: InMemorySource = InMemorySource(document) # Connect a data source to your schema.

executor = InMemoryExecutor(sources=[source], indices=[document_index]) # Tie it all together to run your configuration.
app = executor.run()

source.put([{"id": "happy_dog", "body": "That is a happy dog"}])
source.put([{"id": "happy_person", "body": "That is a very happy person"}])
source.put([{"id": "sunny_day", "body": "Today is a sunny day"}])

print(app.query(query, query_text="Who is a positive friend?")) # Run your query.
```

Ready to go to production? We are launching our first Vector DB connectors soon! [Tell us which Vector DB we should support!](https://github.com/superlinked/superlinked/discussions/41)

## Articles

- [Vector DB Comparison](https://superlinked.com/vector-db-comparison/): Open-source collaboritve comparison of vector databases by Superlinked.
- [Vector Hub](https://superlinked.com/vectorhub/): VectorHub is a free and open-sourced learning hub for people interested in adding vector retrieval to their ML stack

## Support

If you encounter any challanges during your experiments, feel free to create an [issue](https://github.com/superlinked/superlinked/issues/new?assignees=ClaireSuperlinked&labels=bug&projects=&template=bug_report.md&title=), request a [feature](https://github.com/superlinked/superlinked/issues/new?assignees=ClaireSuperlinked&labels=enhancement&projects=&template=feature_request.md&title=) or to [start a discussion](https://github.com/superlinked/superlinked/discussions/new/choose).
Make sure to group your feedback in separate issues and discussions by topic. Thank you for your feedback!

