GCP: Publish Python Package in Production
This guide explains how to use Google Artifact Registry to manage shared Python code as a package. This approach eliminates code duplication between your Cloud Functions and server.
Step 1: Structure Your Shared Code
Create a new Python package for your shared logic (e.g., common_logic).
common_logic/
├── setup.py
├── common_logic/
│ ├── __init__.py
Step 2: Create setup.py
Define your package configuration in a setup.py file:
from setuptools import setup, find_packages
setup(
name="common_logic",
version="0.1.0",
packages=find_packages(),
install_requires=[
"pandas>=1.3.0",
],
author="Your Name",
author_email="[email protected]",
description="Common logic for app",
)
Step 3: Set Up Google Artifact Registry
- Enable the Artifact Registry API:
gcloud services enable artifactregistry.googleapis.com
- Create a Python repository:
gcloud artifacts repositories create python-packages \
--repository-format=python \
--location=us-central1 \
--description="Python packages repository"
Step 4: Configure Authentication
- Create a service account:
gcloud iam service-accounts create artifact-publisher \
--description="Service account for publishing to Artifact Registry"
- Grant necessary permissions:
gcloud artifacts repositories add-iam-policy-binding python-packages \
--location=us-central1 \
--member="serviceAccount:artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com" \
--role="roles/artifactregistry.writer"
- Create and download a key:
gcloud iam service-accounts keys create key.json \
--iam-account=artifact-publisher@${PROJECT_ID}.iam.gserviceaccount.com
Step 5: Build and Upload Package
- Install build tools:
pip install build twine
- Build the package:
python -m build
- Configure
twinefor Artifact Registry:
cat > ~/.pypirc << EOL
[distutils]
index-servers = common-logic-repo
[common-logic-repo]
repository: https://us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/
username: _json_key_base64
password: $(base64 -w0 key.json)
EOL
- Upload the package:
twine upload --repository common-logic-repo dist/*
Step 6: Use the Package
In Cloud Functions
- Create a
requirements.txtfile:
--index-url https://pypi.org/simple
--extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
common-logic==0.1.0
- Use the package in your Cloud Function:
from common_logic import ...
def cloud_function(request):
# Your cloud function code using the imported functions
pass
In Server Code
- Add to your server's
requirements.txt:
--index-url https://pypi.org/simple
--extra-index-url https://oauth2accesstoken:${ARTIFACT_REGISTRY_TOKEN}@us-central1-python.pkg.dev/${PROJECT_ID}/python-packages/simple/
common-logic==0.1.0
- Use it in your server code:
from common_logic import ...
# Your server code using the imported functions
Step 7: CI/CD Integration
- Add the service account key as a secret in your GitHub repository.
- Update your Cloud Build configuration:
steps:
- name: 'python'
entrypoint: pip
args: ['install', 'build', 'twine']
- name: 'python'
entrypoint: python
args: ['-m', 'build']
dir: 'common_logic'
- name: 'python'
entrypoint: twine
args: ['upload', '--repository', 'common-logic-repo', 'dist/*']
dir: 'common_logic'
env:
- 'TWINE_USERNAME=_json_key_base64'
- 'TWINE_PASSWORD=${_ARTIFACT_REGISTRY_KEY}'
Step 8: Version Management
- Update the version in
setup.py. - Build and upload the new version.
- Update
requirements.txtin both Cloud Functions and server code. - Deploy both components.
Best Practices
- Use semantic versioning for your package.
- Pin specific versions in
requirements.txt. - Test new versions thoroughly before deploying.
- Keep a changelog of version changes.
- Use environment variables for
PROJECT_IDandLOCATION. - Include comprehensive documentation in your package.
Common Issues and Solutions
Authentication Errors
- Verify service account permissions.
- Ensure
key.jsonis properly encoded. - Check
.pypircconfiguration.
Package Not Found
- Verify repository URL format.
- Check if the package was successfully uploaded.
- Ensure
requirements.txtuses the correct URL format.
Version Conflicts
- Pin specific versions of dependencies.
- Use virtual environments for testing.
- Document dependency requirements clearly.
SOCIAL SHARE CARD GENERATOR