Skip to main content
Superblocks supports common Python and JavaScript libraries in backend APIs. If you’re self-hosting data plane, you can also install your own custom libraries from public and private package registries, or by using an install script.

Add custom dependencies and Dockerfile

To get started building a custom image, create a new directory for your custom agent (e.g. /custom_js_packages or /custom_python_packages). In this directory, add your package dependency files and custom Dockerfile, following the instructions below for JavaScript and Python.

JavaScript

Public registry

To add custom JavaScript packages from a public registry, create a package.json file that includes the packages to install. For example:
{
  "dependencies": {
    "shelljs": "^0.8.3",
    "xlsx": "^0.18.5"
  }
}
Next, create a Dockerfile with the following contents:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

# Install Nodejs + NPM

ENV NODE_VERSION=20.16.0
RUN apt-get update && apt-get install -y curl gnupg
RUN curl -fsSL https://deb.nodesource.com/setup_$NODE_VERSION.x | bash - && \
  apt-get install -y nodejs && \
  npm install -g [email protected] && \
  apt-get clean && rm -rf /var/lib/apt/lists/*

# Verify installation

RUN node -v && npm -v
WORKDIR /app
COPY ./package*.json ./
RUN npm install
RUN mv node_modules/* /app/worker.js/node_modules/

WORKDIR /custom_js_packages
COPY ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/
If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent for ${SUPERBLOCKS_IMAGE_NAME}. Otherwise, use ghcr.io/superblocksteam/agent.For ${SUPERBLOCKS_IMAGE_TAG} use the desired agent version, or latest (min. supported version is v1.7.3).

Private registry

To add JavaScript packages from a private registry, include these packages in your package.json. Also add a .npmrc file with the URL(s) to your private registry and your authentication credentials. For example:
//${PRIVATE_REGISTRY_URL}:_authToken=${AUTH_TOKEN}
@myorg:registry=https://${PRIVATE_REGISTRY_URL}
@myparentorg:registry=https://${PRIVATE_REGISTRY_URL}
Next, create a Dockerfile that includes the .npmrc file in the COPY command:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

# Install Nodejs + NPM

ENV NODE_VERSION=20.16.0
RUN apt-get update && apt-get install -y curl gnupg
RUN curl -fsSL https://deb.nodesource.com/setup_$NODE_VERSION.x | bash - && \
  apt-get install -y nodejs && \
  npm install -g [email protected] && \
  apt-get clean && rm -rf /var/lib/apt/lists/*

# Verify installation

RUN node -v && npm -v
WORKDIR /app
COPY ./package*.json ./
RUN npm install
RUN mv node_modules/* /app/worker.js/node_modules/

WORKDIR /custom_js_packages
COPY ./.npmrc ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/

Python

Public registry

To add custom Python packages from a public registry, create a requirements.txt file that includes the packages to install. Next, create a Dockerfile with the following contents:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY requirements.txt custom-requirements.txt
RUN pip3 install --no-cache-dir -r custom-requirements.txt

Private registry

To add Python packages from a private registry, include these packages in your requirements.txt file. In your Dockerfile, also include an extra index URL for your private repository:
FROM ${SUPERBLOCKS_IMAGE_NAME}
COPY requirements.txt custom-requirements.txt
RUN pip3 install -r custom-requirements.txt --extra-index-url ${PRIVATE_REGISTRY_URL}

Manually install

Python packages with a setup.py script can be installed by creating a wheel, copying it to your Docker image, and installing it using pip. Update your Dockerfile to include:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY PATH/TO/PACKAGE.whl PACKAGE.whl
RUN pip3 install --no-cache-dir PACKAGE.whl

Build, publish, and deploy custom image

Once all dependencies are added and the Dockerfile is created, build and tag the new image:
docker buildx build --platform linux/amd64 -f Dockerfile . -t ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
After building the image, push the new image tag to your container registry:
docker push ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
With the image available in your container registry, configure your deployment to pull your custom image, and redeploy:
Update the following value in your agent’s Terraform file
superblocks_agent_image = ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}"