Customize Packages
Superblocks supports common Python and JavaScript libraries in backend APIs. If you're running the On-Premise Agent, you can also install your own custom libraries from public and private package registries, or by using an install script.
Add custom dependencies and Dockerfile
To get started building a custom image, create a new directory for your custom agent (e.g. /custom_js_packages
or /custom_python_packages
). In this directory, add your package dependency files and custom Dockerfile
, following the instructions below for JavaScript and Python.
JavaScript
Public registry
To add custom JavaScript packages from a public registry, create a package.json
file that includes the packages to install. For example:
{
"dependencies": {
"shelljs": "^0.8.3",
"xlsx": "^0.18.5"
}
}
Next, create a Dockerfile
with the following contents:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}
WORKDIR /custom_js_packages
COPY ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/
If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent
for ${SUPERBLOCKS_IMAGE_NAME}
. Otherwise, use ghcr.io/superblocksteam/agent
.
For ${SUPERBLOCKS_IMAGE_TAG}
use the desired agent version, or latest
(min. supported version is v1.7.3
).
Private registry
To add JavaScript packages from a private registry, include these packages in your package.json
. Also add a .npmrc
file with the URL(s) to your private registry and your authentication credentials. For example:
//${PRIVATE_REGISTRY_URL}:_authToken=${AUTH_TOKEN}
@myorg:registry=https://${PRIVATE_REGISTRY_URL}
@myparentorg:registry=https://${PRIVATE_REGISTRY_URL}
${PRIVATE_REGISTRY_URL}
is your private registry URL (e.g. "npm.pkg.github.com"), and ${AUTH_TOKEN}
is the token used for authenticating with your private registry. Reference your package registry’s specific instructions for how to set up authentication.
Next, create a Dockerfile
that includes the .npmrc
file in the COPY
command:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}
WORKDIR /custom_js_packages
COPY ./.npmrc ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/
Python
Public registry
To add custom Python packages from a public registry, create a requirements.txt
file that includes the packages to install. Next, create a Dockerfile
with the following contents:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}
WORKDIR /custom_python_packages
COPY requirements.txt custom-requirements.txt
RUN pip3 install --no-cache-dir -r custom-requirements.txt
If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent
for ${SUPERBLOCKS_IMAGE_NAME}
. Otherwise, use ghcr.io/superblocksteam/agent
.
For ${SUPERBLOCKS_IMAGE_TAG}
use the desired agent version, or latest
(min. supported version is v1.7.3
).
Private registry
To add Python packages from a private registry, include these packages in your requirements.txt
file.
In your Dockerfile
, also include an extra index URL for your private repository:
FROM ${SUPERBLOCKS_IMAGE_NAME}
COPY requirements.txt custom-requirements.txt
RUN pip3 install -r custom-requirements.txt --extra-index-url ${PRIVATE_REGISTRY_URL}
Reference your package registry’s instructions for how to set up authentication.
Manually install
Python packages with a setup.py
script can be installed by creating a wheel, copying it to your Docker image, and installing it using pip
.
Update your Dockerfile
to include:
FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}
WORKDIR /custom_python_packages
COPY PATH/TO/PACKAGE.whl PACKAGE.whl
RUN pip3 install --no-cache-dir PACKAGE.whl
Build, publish, and deploy custom image
Once all dependencies are added and the Dockerfile
is created, build and tag the new image:
docker buildx build --platform linux/amd64 -f Dockerfile . -t ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
After building the image, push the new image tag to your container registry:
docker push ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}
With the image available in your container registry, configure your On-Premise Agent to pull your custom image, and redeploy the agent:
- Terraform
- Kubernetes
- Docker
Update the following value in your agent's Terraform file
superblocks_agent_image = ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}"
Update the following value in your agent's values.yaml
image:
repository: ${IMAGE_REGISTRY}/${IMAGE_NAME}
pullPolicy: IfNotPresent
tag: ${IMAGE_TAG}
Update the following value in your agent's compose.yaml
services:
#(...)
agent:
image: ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}