or
or
By clicking below, you agree to our terms of service.
New to HackMD? Sign up
Syntax | Example | Reference | |
---|---|---|---|
# Header | Header | 基本排版 | |
- Unordered List |
|
||
1. Ordered List |
|
||
- [ ] Todo List |
|
||
> Blockquote | Blockquote |
||
**Bold font** | Bold font | ||
*Italics font* | Italics font | ||
~~Strikethrough~~ | |||
19^th^ | 19th | ||
H~2~O | H2O | ||
++Inserted text++ | Inserted text | ||
==Marked text== | Marked text | ||
[link text](https:// "title") | Link | ||
 | Image | ||
`Code` | Code |
在筆記中貼入程式碼 | |
```javascript var i = 0; ``` |
|
||
:smile: | ![]() |
Emoji list | |
{%youtube youtube_id %} | Externals | ||
$L^aT_eX$ | LaTeX | ||
:::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Syncing
xxxxxxxxxx
Valerio I have moved this to https://hackmd.io/@conda-us-rse-2023/tutorial
Please edit that version
Valerio I have moved this to https://hackmd.io/@conda-us-rse-2023/tutorial
Please edit that version
Valerio I have moved this to https://hackmd.io/@conda-us-rse-2023/tutorial
Please edit that version
Valerio I have moved this to https://hackmd.io/@conda-us-rse-2023/tutorial
Please edit that version
Publish Your Software in conda-forge
US RSE-23 Tutorial
Valerio Maggio and Dave Clements, Anaconda
October 10, 2023
This tutorial was presented at PyCon 2023 US. This is a copy that we are updating to reflect what we learned at PyCon. This is a PyCon US 2023 sprint project.
The original, unmodified tutorial is here.
What's been changed so far:
__init.py__
- decide what functions should go here. (DPC)Before we start
In this hands-on tutorial, we will be assuming that:
A word for Windows users
If you have the Linux Subsystem for Windows installed then you can use the mainline instructions in this tutorial. You will only need to follow the "Windows specific" instructions if you don't have LSW installed.
When instructions are different on Windows than they are on Linux, macOS, or LSW there will be a separate section with Windows instructions like this:
Windows (non LSW) 💁
Launch the command prompt.
We'll be using the
command prompt
and then theanaconda prompt
throughout.Getting started
Before diving into the exercises, let's make sure we have everything we need to get started.
conda
andgit
installed on your computer.If you don't have all of these then please have a look at the next section for detailed instructions on how to install what you need.
Otherwise, please feel free to jump directly to the Create Conda Environment section to proceed.
Set up your computer
1. Install Miniconda
For this tutorial we recommend Miniconda, a free minimal installer for
conda
. It includes only conda, Python, the packages they depend on, and a small number of other useful packages, includingpip
,zlib
and a few others.Use the
conda install
command to install over 3000 additional conda packages from the Anaconda repository, and over 20000 packages from conda-forge.Please download and install the most recent Miniconda installer that matches your computer architecture and Operating System. Accept the default settings when installing.
2. Install Git
Installing
Git
will vary depending on your Operating System. You could follow all the instructions reported here for a detailed step-by-step setup.3. Code Editor
For these very reason, we are not requiring you to use any specific code editor. Any editor that you like which has syntax highlighting for the Python language will do.
During the live coding sessions of the tutorial, we will be using Visual Studio Code, with the MS Python Extension.
Create Conda Environment
If you are here, it's because you already have a working version of
conda
installed on your computer. The first thing to do now is to create a virtual Conda environment that we will be using throughout the tutorial.Windows (non LSW) 💁
Launch the newly installed Anaconda Prompt:
To create our new
packaging
conda environment just type the following instruction in the terminal/command line/Anaconda Prompt:This will create a new conda environment using Python 3.10, and then install
IPython
andpytest
as extra packages.This is all we need to get started. At this point, we just have to activate our new environment, and then we will be ready to go!
The command line now starts with
(packaging)
instead of(base)
reflecting that you now in thepackaging
environment.D&D ROLLER
Image Not Showing
Possible Reasons
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →In this tutorial we will be working together on a new and never-been-seen-before application that will be implement a die roller to be used during our campaign of Dungeons & Dragons.
The specs of this applications are pretty simple:
We would need to roll a single die, choosing one of
d4
,d6
,d8
,d10
,d12
,d20
,d100
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Learn More →We would need to make multiple die rolls, of multiple dice (e.g.
2d4
and3d6
).Nice, let's get started! Shall we!?
1. Create & Clone the
dnd-roller
GitHub repositoryLet's start by creating the GitHub repository that will host our
dnd-roller
project.Let's go to your GitHub profile page (i.e.
https://github.com/<your-gh-username>
); click the green 🖥️ New button in the upper right corner.Name the new repository dnd-roller, with the following description:
Also:
README
filegitignore
, choosing the template for PythonMIT License
We're now ready to proceed, and to hit the button Create repository.
At this point, all that remains is to clone this repo on your local computer:
Image from teeturtle.com
2. Let's dig into our
dnd-roller
It's finally time to write some Python code for our
dnd-roller
!We will start by creating the new
dnd_roller
Python package, and its corresponding folder fortests
.Note the underscore in the
dnd_roller
subdirectory name. The directory of your project will look something like this:The
dice.py
ModuleLet's now work on our new
dice.py
module:dnd_roller/dice.py
. This module will contain all the functions that implement the main core functionalities of ourdnd-roller
.Let's recap what we need to implement in our dice roller:
roll
)d4|6|8|10|12|100
dice_roll
)"2d4, 3d6, 1d10, 6d12"
) and will generate a tabular report of the outcomes.Here is the code for
dice.py
. Paste this into a file nameddnd_roller/dice.py
using your text editor.Here is more information about this module. We will use this to implement tests.
1. The
roll
functionThe requirements for this function are pretty simple indeed:
roll(d=4) -> return a number in [1,4]
roll(d=10) -> return a number in [1,10]
roll(d=5) -> Exception: Unsupported die
roll(d=100) -> roll(d=10) * 10
💡 A few comments:
In this implementation, we first want to be sure that the parameter
d
does actually correspond to an integer. We do so adopting an idiomatic approach known asEAFP
(Easier to ask for Forgiveness than Permission). Afterwards, we check that the number of sides chosen for the die is indeed valid in D&D, and then we simply rely on therandom.randint
function to generate the result of our rolling. Simple as that!2. The
dice_roll
functionGiven that all the heavy-lifting has been done already in the
roll
function, the implementation of thedice_roll
function is pretty straightforward: it just needs to call theroll
function multiple times, and return the outcomes as a Python list.3. The
sequence_rolls
functionThe last function in
dnd-roller
issequence_rolls
.Generating the dice rolls is pretty straightforward. All that is really left to do is to parse the input sequence so that it becomes input parameters for the
dice_roll
function. However that is pretty simple to do as well, so we do it in a quite convoluted way using a combination of functional programming and dictionary comprehension to make it fun.__init__.py
We'll need an
__init__.py
file to make Python treat our directory as a package. The file can be empty, but it must exist. Create it in the maindnd_roller
directory.__init__.py
is run once when the module is first imported and it's an important part of defining the public interface to our module.Our module has three functions,
roll
,dice_roll
, andsequence_rolls
. Theroll
function supports thedice_roll
function, which in turn supports thesequence_rolls
function. So, it is possible that we want onlysequence_rolls
in the package's public interface. But, these 3 functions all serve useful functions so, let's put all 3 in the public interface. Create__init_py__
in our top level directory:which enables this
instead of this
Users of your package don't need to know the details of where you implemented your API.
Dependencies and
tabulate
As for the tabulation part, we will leverage the
tabulate
package, that is directly available in the default conda channel:Alternatively, you could install the
tabulate
package directly from PyPI:💡 This is a good opportunity to remind you that it is indeed possible to install non-conda packages into a Conda environment 💫
Let's try it out
🎉 We are finally done with our fancy
dnd-roller
. All we need to do is to try to generate some tabular report. In a [I]Python interpreter:We could try the new implementation interactively in the Python interpreter.
Note:
ipython
(or even defaultpython
interpreter) would equally do:Testing
Published packages (and all software for that matter) should include test cases.
To verify that this implementation does everything we expect it to, the best possible way is to write a few tests.
We will be using PyTest as our testing framework, so let's first create a
pytest.ini
configuration file in the main root folder. This file will instructpytest
to run the test by managing the namespace resolution properly:roll
testsNow let's create a new test module
test_roll.py
under thetests
folder with a bunch of pretty simple test functions that testroll
:Using pytest, and its
parametrize
feature, we're checking a few corner cases (e.g. NaN inputs, negative or float numbers), as well as correct expected behaviours for ourroll
function (including the only "problematic" case of thed100
).dice_roll
testsTesting should verify that the actual implementation of the
dice_roll
is indeed calling theroll
function (multiple times), without duplicating code! (which is a very bad practice, ed.). To do so, we will be creating aMock
(more here).The more interesting part here is about the tests:
first we could re-use the same strategy we used with
roll
by usingpytest.mark.parametrize
to generate a few test inputs (a.k.a.fixtures
).In this case we will be generating a few combinations of
throws
andsides
values, and we will be checking thatthrows
[1, sides]
sequence_rolls
testsOur last function needs some tests too. We will be adding the tests for the
sequence_rolls
function into a newtests/test_sequence_rolls.py
test module.The first test is pretty similar to the last previously discussed: we're just checking that
sequence_rolls
is not reinventing the wheel, and that thedice_roll
function is called instead, with the right parameters. Again, we are leveraging onunittest.mock.patch
to do so.The other tests are more generally testing the output generated by
sequence_rolls
, so that each sequence has (a) the correct number of rolls, and (b) exactly the very same rolls we are expecting. To do so, we use a trick that sets the random seed. We repeat the calls to random in the same way/order. In this way, we are absolutely sure to always generate the same sequence of numbers.FYI, this is the foundation on which Reproducibility in Data Science could be obtained (e.g. see Reproducibility in Deep learning).
Test!
To run our tests, let's move back to our terminal:
You should get an output similar to the one reported below:
3. Time to pack 📦
It's finally time to pack! Our
dnd-roller
is ready to become a re-usable Python package for everybody to use. So, let's create the skeleton for our future package-to-be, following the instructions reported in the official Python documentation.Creating
setup.py
and package metadataFirst thing we do is to create a new
setup.py
file that uses Pythonsetuptools.setup
to specify initial package metadata:💡 Please note that we specified our external dependencies in the
install_requires=[...]
paramneter of thesetup
function, that istabulate
.❗️ At this point, please feel free to add any additional metadata to the
README.md
file as this will be used for thelong_description
of thednd-roller
package. For example:Finally we will add some additional metadata in the
setup.cfg
andpyproject.toml
configuration files, such as the license, and the package classifiers.💡 Note: We will soon see how these metadata can be consumed by automatic build tools for packaging.
Similarly, in
pyproject.toml
:🎉 Whoot whoot! Now everything is ready for our
dnd-roller
package!Publish on GitHub
Time to publish everything on GitHub:
If we now open the browser, and visit the GH repository url (e.g., https://github.com/leriomaggio/dnd-roller), you should see something similar to the image below.
4. By the power of Grayskull… I have the Conda recipe
Image Credits: https://he-man.fandom.com/
All the hipster geeks in the audience shouldn't require further references and explanation.
And Yes: you are still in the right room! We are still talking about Conda and Python packaging.
From the official documentation:
🎉 Looks like a fantastic treat! We will be using
grayskull
to automatically generate a recipe for ourdnd-roller
project, so that we can build a conda project for it!Installing
Grayskull
We start by installing
grayskull
usingconda
, from theconda-forge
channel:Creating a Release on GitHub
The next thing we want to do, is to create a release of our project on GitHub. Once we have done that, we will be able to use
grayskull
to generate our recipe. In fact, grayskull will fetch all the necessary information (and package) from GitHub to prepare ourconda-recipe
.To do a relase, we can use the GitHub interface directly. The only thing to bear in mind is to specify a proper version tag for our release:
v0.0.1
. The version tag is whatgrayskull
will be using to gather the version of our package, as well as the name of the archive generated by GitHub. Click Create a new release on the right hand side of your GitHub's repo page, and then create the releaseOnce we have a release, we are able to proceed to generate our
conda-recipe
with Grayskull.Generating conda recipe for
dnd-roller
One of the main advantages of using
grayskull
is not only that we don't need to worry about (manually) creating theconda-recipe
to build our package, but it can also get everything that is required directly from GitHub.Generate the recipe for
dnd-roller
withgrayskull
is just two-steps away:dnd-roller
folder, please move away (say, in the parent directory), and create a new folder namedgrayskull
(or as you prefer, the name does not matter):grayskull
folder, and generate the recipe:🎉 When it's completed, you should now see a
dnd-roller
folder, containing ameta.yml
.This is indeed your conda recipe we where hoping for! 👨🍳
5. It's time for
conda build
Now that we have our recipe, all that's need to do is to use it to build our
dnd-roller
conda package.First install
conda-build
which will enable theconda build
command.Next, still within the
grayskull
folder, let's type:If everything goes well, a
dnd-roller.tar.gz
archive should have been created hereWindows 💁
6. Submitting to conda-forge (time allowing)
Today, we are not going to submit multiple (or even single) copies of our
dnd-roller
package to conda-forge. We don't want to test the patience of the conda-forge gods.But, we will get you to the point just before submission, and time allowing, we will also show you how to create your own channel on anaconda.org and publish packages there.
conda-forge's step-by-step instructions
The instructions here are heavily based on the conda-forge instructions for package submission.
From conda-forge
We got this!
We will fork the repo using the GitHub web interface, and then clone that fork on our laptop
Now, clone the new repo locally.
That clone may test the Salt Palace wifi - the repo is around 100mb.
We could do this, but
grayskull
has already generated a perfectly goodmeta.yaml
that we can use, so let's use that instead.Windows 💁
Some things to note:
meta.yaml
file generated bygrayskull
. See below.staged-recipes/recipes/example/meta.yaml
file is full of useful guidance, as is the conda-forgemeta.yml
documentation. Spend some time getting to understand the contents of this file.Now, lets tidy up the
meta.yaml
file we just copied in. The end of that file says:We need to replace
AddYourGitHubIdHere
with our GitHub ID.Thanks to the power of Grayskull we already have a SHA256.
Thanks to the power of Valerio we have already created our tests.
The file generated by Grayskull contains no comments.
conda-forge's checklist
The conda-forge documentation follows the above instructions with this checklist:
Our
meta.yaml
saysMIT
andMIT
is on the example list of approved strings, so we are good.Some of the packages that are merged into conda-forge have a
LICENSE.txt
file alongside themeta.yaml
file, and some don't.Does the MIT license require this? We have no idea, but this recently merged PR uses an MIT license and it does not include a top level
LICENSE.txt
file. If they don't need one, we don't either!Already done.
Already done.
Our folder is
dnd-roller
and that does not collide in anyway withexample
which is the only other folder in therecipes
directory.Commit, push, request
Let's get our staged recipe into our GitHub repo, and then submit a pull request to conda-forge.
The status message in the push tells us where to go next:
Go there to create (almost) a conda-forge PR submissions. conda-forge PRs use this PR template:
We have a decidely Pythonic submission.
Next there is a checklist. Read it and add
x
's as you confirm each item.When it's ready, click Create pull request to submit it. EXCEPT DON'T DO THAT TODAY.
After submitting the PR
There is a whole post-staging process for what happens to your PR after submission.
conda-forge is an all volunteer organization, and depending on how much else is going on, it may take a while for the conda-forge team to engage with your PR. After the initial ping to the appropriate team (see above), how long should you wait before pinging conda-forge again? If you can, try to wait at least a week, and always (always) be polite in your communications.
Publishing in conda-forge
From the conda-forge doc:
7. Publishing in your own channel (time allowing)
You, as Valerio has done, can also publish your package in your own channel on Anaconda.org.
These instructions are based on the conda documentation.
Create an account on Anaconda.org
Install anaconda-client:
conda install anaconda-client
Login
anaconda login
Upload your package to Anaconda.org:
Windows 💁
Now, go to your profile page on anaconda.com, which is at https://anaconda.org/YOUR_USERNAME.