S3pypi installation
Install the s3pypi command line tool by running
{% c-block language="ini" %}
$ (sudo) pip install -U s3pypi
{% c-block-end %}
in your console. If everything goes well, you should be able to run the s3pypi command line tool now:
{% c-block language="js" %}
$ s3pypi -v
{% c-block-end %}
Using S3pypi to publish a package
Now you’re ready to publish your first Python package to your private repository. Make sure you have your AWS credentials set up in your environment, and that you have permission to upload files to the S3 bucket that you created in the previous step.
In order to upload your package to your repository, cd to the root directory of your project, and run:
{% c-block language="js" %}
$ s3pypi --bucket pypi.example.com
{% c-block-end %}
For this to work, your project should contain a setup script. The official Python documentation contains a detailed guide on creating the setup script, and on distributing packages in general.
Installing packages
Install your packages using pip by pointing the --extra-index-url to your subdomain:
{% c-block language="js" %}
$ pip install my-project --extra-index-url https://pypi.example.com/
{% c-block-end %}
Alternatively, you can configure the index URL in ~/.pip/pip.conf:
{% c-block language="js" %}
[global]
extra-index-url = https://pypi.example.com/
{% c-block-end %}
Security
Access control for publishing files to your brand-new pypi repository is regulated entirely using your AWS Identity and Access Management. You could give an IAM user publish rights by assigning the managed IAM policy “PublishS3PyPIPackages” that is created by Cloudformation to their user profile.
Pip supports basic authentication for authenticating against a private PyPI server. Unfortunately, S3 does not. This increases the risk of your private packages leaking. To reduce this risk, you can take some additional measures:
- s3pypi supports adding a --secret SECRET parameter when publishing a package. This allows you to add a random string of your desired length to the url to obfuscate the location of the private packages. When you use this option, don’t forget to also update the extra-index-url in your config file to https://pypi.example.com/SECRET/.
- If you have a (set of) static IP(s), you can add IP whitelisting to the Web application firewall (WAF) of your cloudfront distribution, to only allow pip clients from this set of IP addresses to download packages. By using this template instead of the one above, the WAF is getting configured auto-magically together with the Cloudfront distribution and S3 bucket.
If these security measures are insufficient for your needs, you could take a look at the open source project s3auth.com, but you should also consider hosting your Python repository elsewhere.
All done? We hope private packages will never make you say ni again…
