Self-host Sourcebot
Sourcebot is open source and can be self-hosted using our official Docker image.
Quick Start Guide
Create a config
By default, Sourcebot requires a configuration file with a list of code host connections that specify what repositories should be synced (cloned and indexed). To get started, run the following command to create a starter config.json
:
This config creates a single GitHub connection named starter-connection
that specifies Sourcebot as a repo to sync.
Launch your instance
Sourcebot is packaged as a single Docker image. In the same directory as config.json
, run the following command to start your instance:
Navigate to localhost:3000
to start searching the Sourcebot repo.
Link your code
Sourcebot supports indexing public & private code on the following code hosts:
Architecture
Sourcebot is shipped as a single docker container that runs a collection of services using supervisord:
Sourcebot consists of the following components:
- Web Server : main Next.js web application serving the Sourcebot UI.
- Backend Worker : Node.js process that incrementally syncs with code hosts (e.g., GitHub, GitLab etc.) and asynchronously indexes configured repositories.
- Zoekt : the open-source, trigram indexing code search engine that powers Sourcebot under the hood.
- Postgres : transactional database for storing business-logic data.
- Redis Job Queue : fast in-memory store. Used with BullMQ for queuing asynchronous work.
.sourcebot/
cache : file-system cache where persistent data is written.
You can use managed Redis / Postgres services that run outside of the Sourcebot container by providing the REDIS_URL
and DATABASE_URL
environment variables, respectively. See the configuration for more configuration options.
Scalability
One of our design philosophies for Sourcebot is to keep our infrastructure radically simple while balancing scalability concerns. Depending on the number of repositories you have indexed and the instance you are running Sourcebot on, you may experience slow search times or other performance degradations. Our recommendation is to vertically scale your instance by increasing the number of CPU cores and memory.
Sourcebot does not support horizontal scaling at this time, but it is on our roadmap. If this is something your team would be interested in, please contact us at team@sourcebot.dev.
Telemetry
By default, Sourcebot collects anonymized usage data through PostHog to help us improve the performance and reliability of our tool. We don’t collect or transmit any information related to your codebase. In addition, all events are sanitized to ensure that no sensitive details (ex. ip address, query info) leave your machine.
The data we collect includes general usage statistics and metadata such as query performance (e.g., search duration, error rates) to monitor the application’s health and functionality. This information helps us better understand how Sourcebot is used and where improvements can be made.
If you’d like to disable all telemetry, you can do so by setting the environment variable SOURCEBOT_TELEMETRY_DISABLED
to true
:
If you disabled telemetry correctly, you’ll see the following log when starting Sourcebot: