Skip to main content

Customize Packages

Superblocks supports common Python and JavaScript libraries in backend APIs. If you're running the On-Premise Agent, you can also install your own custom libraries from public and private package registries, or by using an install script.

Add custom dependencies and Dockerfile

To get started building a custom image, create a new directory for your custom agent (e.g. /custom_js_packages or /custom_python_packages). In this directory, add your package dependency files and custom Dockerfile, following the instructions below for JavaScript and Python.

JavaScript

Public registry

To add custom JavaScript packages from a public registry, create a package.json file that includes the packages to install. For example:

{
"dependencies": {
"shelljs": "^0.8.3",
"xlsx": "^0.18.5"
}
}

Next, create a Dockerfile with the following contents:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_js_packages
COPY ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/
info

If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent for ${SUPERBLOCKS_IMAGE_NAME}. Otherwise, use ghcr.io/superblocksteam/agent.

For ${SUPERBLOCKS_IMAGE_TAG} use the desired agent version, or latest (min. supported version is v1.7.3).

Private registry

To add JavaScript packages from a private registry, include these packages in your package.json. Also add a .npmrc file with the URL(s) to your private registry and your authentication credentials. For example:

//${PRIVATE_REGISTRY_URL}:_authToken=${AUTH_TOKEN}
@myorg:registry=https://${PRIVATE_REGISTRY_URL}
@myparentorg:registry=https://${PRIVATE_REGISTRY_URL}
info

${PRIVATE_REGISTRY_URL} is your private registry URL (e.g. "npm.pkg.github.com"), and ${AUTH_TOKEN} is the token used for authenticating with your private registry. Reference your package registry’s specific instructions for how to set up authentication.

Next, create a Dockerfile that includes the .npmrc file in the COPY command:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_js_packages
COPY ./.npmrc ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/

Python

Public registry

To add custom Python packages from a public registry, create a requirements.txt file that includes the packages to install. Next, create a Dockerfile with the following contents:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY requirements.txt custom-requirements.txt
RUN pip3 install --no-cache-dir -r custom-requirements.txt
info

If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent for ${SUPERBLOCKS_IMAGE_NAME}. Otherwise, use ghcr.io/superblocksteam/agent.

For ${SUPERBLOCKS_IMAGE_TAG} use the desired agent version, or latest (min. supported version is v1.7.3).

Private registry

To add Python packages from a private registry, include these packages in your requirements.txt file. In your Dockerfile, also include an extra index URL for your private repository:

FROM ${SUPERBLOCKS_IMAGE_NAME}
COPY requirements.txt custom-requirements.txt
RUN pip3 install -r custom-requirements.txt --extra-index-url ${PRIVATE_REGISTRY_URL}
info

Reference your package registry’s instructions for how to set up authentication.

Manually install

Python packages with a setup.py script can be installed by creating a wheel, copying it to your Docker image, and installing it using pip.

Update your Dockerfile to include:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY PATH/TO/PACKAGE.whl PACKAGE.whl
RUN pip3 install --no-cache-dir PACKAGE.whl

Build, publish, and deploy custom image

Once all dependencies are added and the Dockerfile is created, build and tag the new image:

docker buildx build --platform linux/amd64 -f Dockerfile . -t ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}

After building the image, push the new image tag to your container registry:

docker push ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}

With the image available in your container registry, configure your On-Premise Agent to pull your custom image, and redeploy the agent:

Update the following value in your agent's Terraform file

superblocks_agent_image = ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}"