tags:work-log

GSOC 2020

Call Schedule:

Monday to Thursday: approx 12:00pm est /18:00pm cst/21:30pm ist
standing/debug: https://zoom.us/j/99567688969

September 7 Zoom Call

  • Fixed Travis
  • Points to ponder:
    1. Tell travis to ignore test failure on new image
    2. Create diff for existing image on master and post to github as comment on PR
    3. Tell CI to generate new baseline on merge-to-master

September 3 Zoom Call

  • Ran Travis tests locally. One error resolved on travis. New errors coming on travis

Goals for monday

  • Debug Travis

September 2 Zoom Call

  • Ran travis tests locally to find the output on the local machine

Goals for tomorrow

  • Write documentation related to the before and after the GSoC changes are done.

September 1 Zoom Call

  • Rebased the PR. One error resolved, one still present.
  • Added documentation

Goals for tomorrow

  • Run test after deleting the cache folder for mpl baseline images

August 31 Zoom Call

  • Fixing travis tests

Goals

  • Run travis tests locally without -n2 flag

August 28 zoom call

  • Added the extraction of basleine image from the mpl cache
  • Did suggested changes

Goals for Monday

  • Do the suggested changes

August 27 Zoom Call

  • Added copy logic to the mpl cache

GOals for tomorrow

  • Add the extraction of basleine image from the mpl cache

August 26 Zoom Call

  • Created user stories

Goals for tomorrow

  • Do suggested changes
  • Copy images to the mpl_cache

August 25 Zoom Call

  • filled the final evaluation by mentee
  • created the plugin
  • checked the plugin

Goals for tomorrow

  • add cache
  • write user stories

August 24 Zoom Call

  • merged two command line statements into one statement
  • fixed the flake8 issues

Goals for tomorrow

  • do remaining suggested pr changes
  • add plugin for image generation

August 20 zoom call

  • fixed flake errors
  • tried to make one command from two commands

Goals for tomorrow

  • make one command from two commands

August 19 Zoom Call

  • tried to fix the doc errors

Goals for tomorrow

  • Fix the flake errors

August 18 zoom call

  • fix flake
  • built the docs

Goals for tomorrow

  • Complete the suggested changes

August 17 zoom call

  • tried to fix travis
  • installed inkspace and graphviz

Goals for tomorrow

  • build docs

August 13 zoom call

  • debugging why travis is not passing: Changes related to recently merged pr with C language changes
  • rectified the conflicts

Goals for monday

  • fix travis
  • write work product

August 12 zoom call

  • debugging why travis is not passing
  • learnt how to run travis test cases through terminal

Goals for tomorrow

  • fix test image discovery with sub-wheels

August 11 zoom call

  • finished blog
  • discovered that installing the sub-wheel means the images don't get found

Goals for tomorrow

  • fix test image discovery with sub-wheels

August 8 zoom call

  • made progress on extending the docs (looks good, needs some copy editing)
  • shortened names
  • goal for week is to get current PR (without deleting images) merged to master, open new PR to work on sorting out CI caching / configuration.

Goals for next call

  • get CI passing on current PR
  • add instructions to release guide about generating and publishing the images wheels

August 6 zoom call

  • content of docs looks good!
  • goal is to get current PR (without deleting images) merged by end of next week

Goals for Friday & Monday

  • re-format docs as bullet points / code blocks
  • add documentation about how to use the sub-package to get (most) of the images
  • rename flag to make shorter

August 5 zoom call

  • went with a simpler approach of generating missing images rather than trying to flag specific tests like suggested yesterday
  • discussed logic of

Goals for the day

  • see if you can make it faster ?
    • how early can we detect that we will not need to copy the result file?
  • write up guide of how to change a test as a developer

August 4 Zoom Call

  • logic
    • in conftest.py "should I mark this node?" via add_marker
    • in decorators.py "did I mark this node?" via get_cloest_marker

goals

  • register a "re-generate this test marker"
  • apply the marker in conf test
  • regenerate or test in the wrapper based on if the marker is attached to the node (aka test)

July 31 Zoom Call

  • Discussed modification for baseline images logic

goals

  • Create command line flag that takes test name as argument and just run the tests mentioned as arguments

July 29 Zoom Call

  • Discussed modification for baseline images logic

goals

  • Create command line flag that takes test name as argument

July 28 Zoom Call

  • Blog Post reviewed and merged
  • Second eval form submitted by mentor and mentee

goals

  • Add CI logic

July 27 Zoom Call

  • done changes suggested in the Blog Post
  • mpl_toolkits image generation flag - working properly

Goals

  • complete the suggestions given in the PRs
  • fill second evaluation for GSoC Form

July 23 Zoom Call

  • wrote blogpost
    • and reviewed
  • debugged why pytest configuration was missing

goals for Monday

  • get test generation / configuration for mpl_toolkits working
  • finish blog post

July 22 Zoom Call

  • Fixed the tests with/without matplotlib_baseline_image_generation
  • create_baseline_images in the test_backend_pgf works fine
  • Discussed about the transition period from nose to pytest
  • Tried to enable pytest work for all directoritories/paths including the mpl_toolkit images.

goals

  • Create Blog Post
  • Fix the tests for image generation with flag in the command line

July 21 Zoom call

  • made progress re-writing the pgf tests to use decorator, but they are failing
  • issue is that the rcparams being set need to be set while the figure is being saved so the context manager and the image test decorator are in conflict

Goals for Tomorrow

  • inject the generate / not generate logic into the pgf compare_figure function

July 20 Zoom call

  • looked at tests in pgf that have thier own compare logic
    • plan is to look at making them use the standard decorator (in seperate PR)
  • test of test code failing
    • plan to adjust their test images to be in the data folder.

July 16 Zoom call

  • made progess on having the test suite copy the files to the correct place (no manual copying needed!)
  • debugged folder creation on the phone
  • centralized the conftest imports

Goals for Monday

  • fix names of generated tests
  • find and fix import errors

July 15 Zoom call

  • working on how to seperate the Matplotlib baseline images and the mpl-toolkit test images

goals for tomorrow

  • automatically copy the files to the right place
  • fix the import error on test discovery

July 14 Zoom call

  • reseaching how to get access to CLI flags

July 13 Zoom call

  • trying to get full test suite running
    • 1652 image comparison tests
    • copied files baseline images
    • had issues with tests that generate more than 1 test image
    • Attempt 1: 1652 failures=> 25 failures => 11 failures => 3 failures => 1 failure => 1 failure => 1 failure => Problem with RMS value. Also, 1 failure was not subset of 3 failures. Last three failures were same.
    • Attempt 2: 1652 => 145 => 143 => 112 =>8 => 1 => RMS value in phase_sprectrum_noise image
    • Attempts aren't related to each other.

plan for tomorrow

  • only try to compare images if we are testing (not generating)
  • copy files to the right place on the initial run.

July 9th Zoom Call

  • got CLI argument working!

Plan for Monday:

  • generate images for all tests
  • complete blog post draft
  • do not try to import the test image package if generating tests

July 8th Zoom Call

  • can run the test to generate the images, copy them, re-run and the tests pass
  • sorted out how to use pytest.mark to select a sub-set of the tests

Plan for tomorrow

  • move the mark inside of the decorator
  • add a flag to either generate baseline images or validate the images are correct

July 7th Zoom Call

Thomas - @SidharthBansal is going to work on writing a bash script that will, in the currently active env, generate the baseline images to put all of the worktree / venv management off for a bit
we need to provide a way from the command line when invoking pytest tell the test suite that it needs to generate the images (rather than testing against them)

July 6th Zoom Call

Discussion about the modification of generation of the baseline images PR like source file should run tests till image comparison point

July 2nd Zoom Call

Troubleshoot azure pipelines errors
PR Reviews

July 1st Zoom Call

PR Review #17557

June 30th Zoom Call

Discussion about the removal of baseline images logic from matplotlib_baseline_images package created in #17557
Suggested changes requested in #17793

June 29th Zoom Call

#17557 PR Review

  • Do Flake8 file changes to ignore the errors in the __init__.py

  • Discussion about Removal of baseline images from the matplotlib_baseline_images package

  • Baseline images may exists or doesn't exist. Focus on the case where the baseline images doesn't exist first. Baseline images are autogenerated from previous version.

  • Sketch of next piece of work:

def create_baseline_images_from_previous_commit():
    # Check that we are in a git repo (e.g. .git directory exists), else raise.
    # Check that git is installed.  (shutil.which)
    # Create virtual env in tmpdir.
    subprocess.run([sys.executable, "-mvenv", tmpenv])
    subprocess.run(["git", "clone", "<current repo>", "<tmpenv/...>"])
    ... git checkout HEAD~
    ... python -mpytest ...
    # -> this will generate result_images in that temporary directory
    # -> copy it and cache it.

June 25th Zoom Call

  • Debugging Travis and appvoyer
  • Learning how to build mpl through ccache or through lto optimisation fastly
  • Use pdb command for debugger console
  • Read pathlib documentation
  • Move the baseline images out of test_data/test_images. They are contained in the image_comparison function.

June 24th Zoom Call

  • Reviewed the GSoC Progress Report 2
  • Shift the install commands to the test_script in appveyor.yml
  • Remove Typographical mistake mpl_toolkits instead of mpl_toolkit
  • Discussed different ways about laying out the packages. Use lib folder instead of the sub-wheels/baseline-images.

June 23rd Zoom Call

June 22nd Zoom Call

  • Add tests/test_data is a separate commit for better visualization of the PR
  • Use Try and except statement in the init.py for mpl and mpl_toolkit
  • Discussed flake8 error

June 18th Zoom Call

  • Rebase the PR and resolve conflicts
  • Change status of the PR from Draft to Ready to review
  • Add try-except block in the decorators so that import works if it is installed. Else it should raise a reasonable exception or fallback to the earlier behaviour

June 17th Zoom Call

Troubleshooting baseline-images package

  • Implement the changes discussed in the meeting and push them
  • Fix git history

June 16th Zoom Call

Troubleshooting the demo package's main package

  • Delete the build before creating the sdist and wheels
  • Use exclude command in the find_packages
  • Put prune dir in the MANIFEST.in
  • Need of result_dir

June 9th Zoom Call

@sidharth - Do we need init.py for baseline-images-package?
@anntzer - No need as baseline-images-package is data specific

@sidharth - Do we need mpl_toolkit_baseline_images ?
@anntzer - We can do it later.

@sidharth - Do we need both README and INSTALL for baseline-images package?
@anntzer - Start with only README with basic documentation

@sidharth - Kindly confirm the locations to symlink baseline images - tools/triage and __image_directory decorator ?
@antony - Yeah, correct

June 5th Gitter meet

@sidharth - Which LICENSE to use in subwheel/mpl-baseline-images-only?
@antony - copy the current LICENSE/LICENSE to the subdirectory

@sidharth - Should I read full tutorials about setuptools?
@antony - Create a test package with two subpackages within same directory

testrepo/
  setup.py
  lib/testpkg/__init__.py
  baseline_images/setup.py
  baseline_images/testdata.txt

-create testrepo.whl, testrepo.tar.gz (the sdist), testrepo-baseline-images.whl, testrepo-baseline-images.tar.gz

@sidharth - do we need 3 packages mpl-test, mpl-no test and mpl-baseline-images?
@antony - No, we will follow following file structure

mpl/
  setup.py
  lib/mpl/...
  lib/mpl/tests/...  [contains the tests .py files]
  baseline_images/
    setup.py
    data/...  [contains the image files]

@sidharth - So, PR#17343 needs to be closed?
@antony - Yes, with reason

@sidharth - What will be the end product after enhancements in baseline-images-only package?
@antony - baseline-images-only package will contain user generated test data and .py files. MPL will not have baseline images[data] in the repo

@sidharth - Should we decentralise mpl-baseline-images and mpl package to two different repositories?
@antony - No, it can lead to coordinatoion problems bw repos

@sidharth - Facing difficulty in understanding sdist, wheels, eggs and binaries
@antony - eggs are obsolete now
sdist(source artefacts) - uncompiled compressed files
wheels(binary artefacts) - compiled compressed file(like zip files) with different extension so that developer's time for compilation is saved

June 2nd call

PR#17434
Make a separate commit just for moving all the baseline image files to avoid really awkward review.

alternative proposal

  1. create test wheel
  2. in thread:
    by doing this [insert instructions] can create a wheel w/ just baseline images
  3. use pypi test to verify:
    https://test.pypi.org
  4. should be trivial to make a conda mpl-test-data:
    https://anaconda.org/conda-forge/mpl_sample_data

why does mpl-test-need to be editible?
records in file that package w/ search path that is then importable
when mpl src tree is edited, gets reflected on import matplotlib
regular install doesn't reflect edits (need to reinstall)

@sidharth:
normal mode does a copy, editable writes file extension?
what's the interaction between mpl & test repos?
- regularly pip just says include directory in search path
- in current layout, when pip installs editable –> images are implicitly installed
- once moved out, can be seperately editably installed, removes implicit problem

e.g. /baseline_images -> symlink to /lib/matplotlib/tests/baseline_images

checkpoints

  • Symlink baseline images out.
  • Create a wheel/sdist with just the baseline images; upload it to testpypi (so that one can do pip install mpl-baseline-images); don't bother removing baseline images from the main repo yet.
  • (later) Update testing machinery to use baselines from mpl-baseline-images rather than from source tree.
Select a repo