vespa

Search, make inferences in and organize vectors, tensors, text and structured data, at serving time and any scale.

This repository contains all the code required to build and run all of Vespa yourself, and where you can see all development as it happens. All the content in this repository is licensed under the Apache 2.0 license.

A new release of Vespa is made from this repository's master branch every morning CET Monday through Thursday.

• Home page: https://vespa.ai
• Documentation: https://docs.vespa.ai
• Continuous build: https://factory.vespa.ai
• Run applications in the cloud for free: vespa.ai/free-trial

Table of contents

• Background
• Install
• Usage
• Contribute
• Building
• License

Background

Use cases such as search, recommendation and personalization need to select a subset of data in a large corpus, evaluate machine-learned models over the selected data, organize and aggregate it and return it, typically in less than 100 milliseconds, all while the data corpus is continuously changing.

This is hard to do, especially with large data sets that need to be distributed over multiple nodes and evaluated in parallel. Vespa is a platform that performs these operations for you with high availability and performance. It has been in development for many years and is used on several large internet services and apps which serve hundreds of thousands of queries from Vespa per second.

Install

Deploy your Vespa applications to the cloud service: console.vespa-cloud.com, or run your own Vespa instance: https://docs.vespa.ai/en/getting-started.html

Usage

• The application created in the getting started guides linked above is fully functional and production-ready, but you may want to add more nodes for redundancy.
• See developing applications on adding your own Java components to your Vespa application.
• Vespa APIs is useful to understand how to interface with Vespa
• Explore the sample applications
• Follow the Vespa Blog for feature updates / use cases
• Join the Vespa Slack community to ask questions and share feedback

Full documentation is at https://docs.vespa.ai.

Contribute

We welcome contributions! See CONTRIBUTING.md to learn how to contribute.

If you want to contribute to the documentation, see https://github.com/vespa-engine/documentation

Building

You do not need to build Vespa to use it, but if you want to contribute you need to be able to build the code. This section explains how to build and test Vespa. To understand where to make changes, see Code-map.md. Some suggested improvements with pointers to code are in TODO.md.

Development environment

C++ and Java building is supported on AlmaLinux 8. The Java source can also be built on any platform having Java 17 and Maven 3.8+ installed. Use the following guide to set up a complete development environment using Docker for building Vespa, running unit tests and running system tests: Vespa development on AlmaLinux 8.

Java environment for Mac

1. Install JDK17, Maven Version Manager and jEnv through Homebrew.

brew install jenv mvnvm openjdk@17

2. For the system Java wrappers to find this JDK, symlink it with

sudo ln -sfn /opt/homebrew/opt/openjdk@17/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-17.jdk

3. Follow "Configure your shell" in https://www.jenv.be. Configuration is shell specific. For zsh use the below commands:

echo 'export PATH="$HOME/.jenv/bin:$PATH"' >> ~/.zshrc
echo 'eval "$(jenv init -)"' >> ~/.zshrc
eval "$(jenv init -)"
jenv enable-plugin export
exec $SHELL -l

4. Add JDK17 to jEnv

jenv add $(/usr/libexec/java_home -v 17)

5. Verify configuration with Maven by executing below command in the root of the source code. Output should refer to the JDK and Maven version specified in the .java-version and mvnvm.properties.

mvn -v

Build Java modules

export MAVEN_OPTS="-Xms128m -Xmx1024m"
./bootstrap.sh java
mvn install --threads 1C

Use this if you only need to build the Java modules, otherwise follow the complete development guide above.

When MAVEN_GLOBAL_SETTINGS is set, bootstrap.sh uses the supplied global Maven settings while generating a wrapper. The generated Maven 3.9.16 distribution is checked against a pinned SHA-256 independently of its download URL.

Run tests for shell scripts (on Mac)

Shell scripts are tested with BATS. To run the tests locally, install the testing framework and its plugins.

brew install node
sudo npm install -g bats bats-assert bats-support bats-mock

Export the BATS_PLUGIN_PATH environment variable to point to the global npm modules directory, which contains the BATS plugins:

export BATS_PLUGIN_PATH="$(npm root -g)"

Then run all tests with the following command (from the root of the repository):

bats -r .

To run a specific test, use:

bats test_dir/test_name.bats

Tests can also be run in IntelliJ IDEA with the BashSupport Pro plugin. Ensure the BATS_PLUGIN_PATH environment variable is exported before launching the IDE to avoid setting it in each run configuration.

License

Code licensed under the Apache 2.0 license. See LICENSE for terms.