Skip to main content

Customize Packages

Superblocks supports common Python and JavaScript libraries in backend APIs. If you're running the On-Premise Agent, you can also install your own custom libraries from public and private package registries, or by using an install script.

Add custom dependencies and Dockerfile​

To get started building a custom image, create a new directory for your custom agent (e.g. /custom_js_packages or /custom_python_packages). In this directory, add your package dependency files and custom Dockerfile, following the instructions below for JavaScript and Python.

JavaScript​

Public registry​

To add custom JavaScript packages from a public registry, create a package.json file that includes the packages to install. For example:

{
  "dependencies": {
    "shelljs": "^0.8.3",
    "xlsx": "^0.18.5"
  }
}

Next, create a Dockerfile with the following contents:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}
 
WORKDIR /custom_js_packages
COPY ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/
info

If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent for ${SUPERBLOCKS_IMAGE_NAME}. Otherwise, use ghcr.io/superblocksteam/agent.

For ${SUPERBLOCKS_IMAGE_TAG} use the desired agent version, or latest (min. supported version is v1.7.3).

Private registry​

To add JavaScript packages from a private registry, include these packages in your package.json. Also add a .npmrc file with the URL(s) to your private registry and your authentication credentials. For example:

//${PRIVATE_REGISTRY_URL}:_authToken=${AUTH_TOKEN}
@myorg:registry=https://${PRIVATE_REGISTRY_URL}
@myparentorg:registry=https://${PRIVATE_REGISTRY_URL}
info

${PRIVATE_REGISTRY_URL} is your private registry URL (e.g. "npm.pkg.github.com"), and ${AUTH_TOKEN} is the token used for authenticating with your private registry. Reference your package registry’s specific instructions for how to set up authentication.

Next, create a Dockerfile that includes the .npmrc file in the COPY command:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_js_packages
COPY ./.npmrc ./package*.json ./
RUN npm install --install-strategy nested --prefix .
RUN mv node_modules/* /app/worker.js/node_modules/

Python​

Public registry​

To add custom Python packages from a public registry, create a requirements.txt file that includes the packages to install. Next, create a Dockerfile with the following contents:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY requirements.txt custom-requirements.txt
RUN pip3 install --no-cache-dir -r custom-requirements.txt
info

If running the agent on Google Cloud Run, use us-east1-docker.pkg.dev/superblocks-registry/superblocks/agent for ${SUPERBLOCKS_IMAGE_NAME}. Otherwise, use ghcr.io/superblocksteam/agent.

For ${SUPERBLOCKS_IMAGE_TAG} use the desired agent version, or latest (min. supported version is v1.7.3).

Private registry​

To add Python packages from a private registry, include these packages in your requirements.txt file. In your Dockerfile, also include an extra index URL for your private repository:

FROM ${SUPERBLOCKS_IMAGE_NAME}
COPY requirements.txt custom-requirements.txt
RUN pip3 install -r custom-requirements.txt --extra-index-url ${PRIVATE_REGISTRY_URL}
info

Reference your package registry’s instructions for how to set up authentication.

Manually install​

Python packages with a setup.py script can be installed by creating a wheel, copying it to your Docker image, and installing it using pip.

Update your Dockerfile to include:

FROM ${SUPERBLOCKS_IMAGE_NAME}:${SUPERBLOCKS_IMAGE_TAG}

WORKDIR /custom_python_packages
COPY PATH/TO/PACKAGE.whl PACKAGE.whl
RUN pip3 install --no-cache-dir PACKAGE.whl

Build, publish, and deploy custom image​

Once all dependencies are added and the Dockerfile is created, build and tag the new image:

docker buildx build --platform linux/amd64 -f Dockerfile . -t ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}

After building the image, push the new image tag to your container registry:

docker push ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}

With the image available in your container registry, configure your On-Premise Agent to pull your custom image, and redeploy the agent:

Update the following value in your agent's Terraform file

superblocks_agent_image = ${IMAGE_REGISTRY}/${IMAGE_NAME}:${IMAGE_TAG}"