-
Notifications
You must be signed in to change notification settings - Fork 5k
devops: Dockerfile and Staging Integration Requirements for CI/CD Pipeline #3491
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
SparshGarg999
wants to merge
2
commits into
openai:main
Choose a base branch
from
SparshGarg999:3489-dockerfile-staging-pipeline
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+228
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| name: Staging Deployment | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - main | ||
|
|
||
| jobs: | ||
| build-and-deploy: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Build Staging Image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| load: true | ||
| tags: openai-python-staging:latest | ||
| target: runner | ||
|
|
||
| - name: Start Staging Container | ||
| run: | | ||
| docker run -d --name staging-app -p 8080:8080 openai-python-staging:latest | ||
|
|
||
| - name: Wait for Staging Health Check | ||
| run: | | ||
| echo "Waiting for container to become healthy..." | ||
| for i in {1..15}; do | ||
| if docker inspect --format='{{json .State.Health.Status}}' staging-app | grep -q "healthy"; then | ||
| echo "Container is healthy!" | ||
| exit 0 | ||
| fi | ||
| sleep 2 | ||
| done | ||
| echo "Container health check timed out." | ||
| docker logs staging-app | ||
| exit 1 | ||
|
|
||
| - name: Run Smoke Tests | ||
| run: | | ||
| echo "Running staging environment smoke tests..." | ||
| RESPONSE=$(curl -s http://localhost:8080/smoke-test) | ||
| echo "Response: $RESPONSE" | ||
| if echo "$RESPONSE" | grep -q '"status": "passed"'; then | ||
| echo "Smoke tests passed successfully!" | ||
| else | ||
| echo "Smoke tests failed!" | ||
| exit 1 | ||
| fi | ||
|
|
||
| - name: Simulate Staging Infrastructure Deployment | ||
| run: | | ||
| echo "Deploying to Kubernetes staging namespace..." | ||
| echo "Autodeploy complete." | ||
|
|
||
| - name: Container Cleanup | ||
| if: always() | ||
| run: | | ||
| docker stop staging-app || true | ||
| docker rm staging-app || true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| # Stage 1: Build dependencies | ||
| FROM python:3.12-alpine AS builder | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Install compiler toolchain and build requirements for potential source builds | ||
| RUN apk add --no-cache gcc musl-dev libffi-dev g++ cargo | ||
|
|
||
| # Create a virtual environment for isolated dependency building | ||
| RUN python -m venv /opt/venv | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
|
|
||
| # Copy package descriptors and source code | ||
| COPY pyproject.toml README.md ./ | ||
| COPY src/ ./src/ | ||
|
|
||
| # Upgrade pip and install package dependencies | ||
| RUN pip install --no-cache-dir --upgrade pip && \ | ||
| pip install --no-cache-dir . | ||
|
|
||
| # Stage 2: Clean, optimized runner image | ||
| FROM python:3.12-alpine AS runner | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| # Install runtime dependencies (like curl if needed, but python built-ins are enough) | ||
| # Copy virtual environment from builder stage | ||
| COPY --from=builder /opt/venv /opt/venv | ||
| ENV PATH="/opt/venv/bin:$PATH" | ||
|
|
||
| # Copy example mock server | ||
| COPY examples/staging_server.py ./staging_server.py | ||
| RUN chmod +x ./staging_server.py | ||
|
|
||
| # Expose staging port | ||
| EXPOSE 8080 | ||
|
|
||
| # Configure default environment variables | ||
| ENV OPENAI_API_KEY="" | ||
|
|
||
| # Health check configuration | ||
| HEALTHCHECK --interval=10s --timeout=5s --start-period=5s --retries=3 \ | ||
| CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8080/healthz')" | ||
|
|
||
| # Command to start the staging server | ||
| CMD ["python", "staging_server.py"] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| # Staging Integration & Docker Deployment Guide | ||
|
|
||
| This document describes how to build, run, and maintain the Docker-based deployment and CI/CD staging pipeline for the `openai` Python SDK. | ||
|
|
||
| --- | ||
|
|
||
| ## 1. Docker Build Instructions | ||
|
|
||
| We use a multi-stage Docker build to optimize image size and maintain compatibility across environments. The final runner stage uses a minimal `python:3.12-alpine` base image. | ||
|
|
||
| ### Building the Image | ||
| To build the Docker image locally: | ||
| ```bash | ||
| docker build -t openai-python-staging:latest . | ||
| ``` | ||
|
|
||
| ### Running the Container | ||
| To run the container, inject the necessary `OPENAI_API_KEY` environment variable: | ||
| ```bash | ||
| docker run -d \ | ||
| --name staging-app \ | ||
| -p 8080:8080 \ | ||
| -e OPENAI_API_KEY="your-api-key-here" \ | ||
| openai-python-staging:latest | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 2. Health Checks & Endpoint Configuration | ||
|
|
||
| The container includes a built-in health check using a Python script to hit the internal HTTP server's `/healthz` endpoint. | ||
| - **Port:** `8080` | ||
| - **Health Check Endpoint:** `/healthz` (returns `200 OK` when healthy). | ||
| - **Smoke Test Endpoint:** `/smoke-test` (imports the SDK and prints the active library version). | ||
|
|
||
| To check the container's health status via Docker: | ||
| ```bash | ||
| docker inspect --format='{{json .State.Health.Status}}' staging-app | ||
| ``` | ||
|
|
||
| --- | ||
|
|
||
| ## 3. Environment Variable Injection | ||
|
|
||
| The container expects the following environment variables: | ||
| - `OPENAI_API_KEY` (Required for API requests). | ||
| - `OPENAI_ORG_ID` (Optional, for specifying organization details). | ||
| - `PORT` (Defaults to `8080` inside the server). | ||
|
|
||
| --- | ||
|
|
||
| ## 4. Rollback & Recovery Procedures | ||
|
|
||
| If a deployment fails the smoke tests or health check in the staging environment, perform the following rollback procedure: | ||
|
|
||
| 1. **Abort Pipeline:** The CI/CD pipeline is configured to fail the step if smoke tests or health checks fail, preventing promotion to production. | ||
| 2. **Revert Deployments:** Redeploy the last known stable image tag: | ||
| ```bash | ||
| kubectl rollout undo deployment/openai-python-staging -n staging | ||
| ``` | ||
| 3. **Logs Verification:** Check logs to identify the build error: | ||
| ```bash | ||
| docker logs staging-app | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,53 @@ | ||
| #!/usr/bin/env python3 | ||
| import http.server | ||
| import json | ||
| import sys | ||
|
|
||
| PORT = 8080 | ||
|
|
||
| class StagingHandler(http.server.BaseHTTPRequestHandler): | ||
| def do_GET(self): | ||
| if self.path == '/healthz': | ||
| self.send_response(200) | ||
| self.send_header('Content-Type', 'application/json') | ||
| self.end_headers() | ||
| self.wfile.write(json.dumps({"status": "healthy", "service": "openai-python-staging"}).encode()) | ||
| elif self.path == '/smoke-test': | ||
| try: | ||
| import openai | ||
| sdk_version = openai.__version__ | ||
| self.send_response(200) | ||
| self.send_header('Content-Type', 'application/json') | ||
| self.end_headers() | ||
| self.wfile.write(json.dumps({ | ||
| "status": "passed", | ||
| "message": "OpenAI SDK imported successfully", | ||
| "version": sdk_version | ||
| }).encode()) | ||
| except Exception as e: | ||
| self.send_response(500) | ||
| self.send_header('Content-Type', 'application/json') | ||
| self.end_headers() | ||
| self.wfile.write(json.dumps({ | ||
| "status": "failed", | ||
| "error": str(e) | ||
| }).encode()) | ||
| else: | ||
| self.send_response(404) | ||
| self.send_header('Content-Type', 'text/plain') | ||
| self.end_headers() | ||
| self.wfile.write(b"Not Found") | ||
|
|
||
| def run(): | ||
| server_address = ('', PORT) | ||
| httpd = http.server.HTTPServer(server_address, StagingHandler) | ||
| print(f"Staging Server running on port {PORT}...") | ||
| try: | ||
| httpd.serve_forever() | ||
| except KeyboardInterrupt: | ||
| print("\nShutting down server.") | ||
| httpd.server_close() | ||
| sys.exit(0) | ||
|
|
||
| if __name__ == '__main__': | ||
| run() |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In the Docker build,
pip install .runs after onlypyproject.tomlandREADME.mdhave been copied into/app. This project’s wheel target packagessrc/openai, so at this point the image has no SDK sources for hatchling to package, and theBuild Staging Imagejob will fail before the container or smoke tests can run. Copysrc/before this install step, or install dependencies separately from the local project.Useful? React with 👍 / 👎.