If you write software in Python, you’ve definitely used pip – Python’s package manager – before. Pip – a recursive acronym for “pip installs packages” – allows you to install Python packages that are publicly available.
Pip defaults to searching in and downloading from PyPI, the official package repository managed by the Python Software Foundation. PyPI is great for publicly available open source packages, such as the one we have open-sourced, but it’s not the go-to choice for distributing private or closed source packages. For that purpose, you should use a privately hosted package repository. There are a number of tools available, such as Gemfury (commercial) or pypiserver (open source). Commercial or open-sourced, all of these options require you to either run your own server or pay for someone else to do that for you.
At November Five, we use many of Amazon’s Web Services, such as EC2, RDS, ElastiCache, SQS, SNS, SES, Route 53, Lambda, DynamoDB, Cloudformation, Cloudfront, and of course, S3. S3 (Simple Storage Service) is an online service for file storage that allows you to store terabytes of data, but can also be used for static hosting. It has the double advantage of being both low in maintenance and cheap; you’d only be paying a couple of cents for the storage and bandwidth used by your site.
The tool we’re open-sourcing, s3pypi, allows you to more easily create your own package repository on S3.
There are a few prerequisites when setting up a Python package repository on S3:
- An AWS account. If you don’t have one already, go sign up.
- A domain or subdomain, e.g. pypi.example.com. You should be able to create or modify the DNS record for the (sub)domain you want to use.
- An SSL certificate for the domain you’re using.
In your AWS account, you need to setup an S3 bucket configured for website hosting, as well as a Cloudfront distribution for serving the content in your S3 bucket over a secure (HTTPS) connection, which is required by pip (by default).
We’ve created a Cloudformation template that configures these resources for you.
- Download the s3pypi template here.
- Upload your SSL certificate to your AWS account and keep note of the ServerCertificateId.
- Open the AWS console in your preferred AWS region, select Cloudformation, and click “Create stack”.
- Give the stack a meaningful name and enter the subdomain and ID of the server certificate.
- Skip through the next steps – unless you want to tag your resources – and create the stack.
- When finished, click the “Outputs” tab, and copy the value of the “CNAMERecordValue” output parameter.
- Create a CNAME (or alias if you’re using Route 53) record for your subdomain and point it to this Cloudfront distribution (CNAMERecordValue).