RAMP-UA questions

tags: Issues Solved ToBeSolved

Document to share issues, ideas, and possible solutions for the RAMP-UA and EcoTwins project (as of beginning of March 2021).
Please feel free to edit/suggest/delete content

Contents

• RAMP-UA questions
  • Contents
  • Assumptions
  • Ideas for further implementations
  • General issues
    • * White spaces in the path not accepted by OpenCL
    • * Another issue
  • TO_DO:

Assumptions

Current assumptions that underline the model

Ideas for further implementations

Once generalized the model for studying the Covid spread to a national scale, we might want to apply to further case studies

• How to implement the calibration on a national scale
• Improve the house-transmission modelling
• Public transport not modelled for now
• Speed up the peed up the population initialisation process

Final agreed folders structure
Taking into account the results of the poll and some deeper understanding of the data, we arrived to the following conclusive figure for the folders structure:

Folders structure poll

Which folders structure you think is best to use for redesigning the model to run on a national scale?

In case you choose a fourth one, please add your suggestions heareafter.
At the moment the edited version of the model runs following the diagram in figure one.

General issues

Try to describe as detailed as possible the issue and steps undertaken to face it

* White spaces in the path not accepted by OpenCL

Description of the problem
The issue arised when changed the path to the OpenCL kernel within opencl/ramp/simulator.py in the class Simulator (the class is called by opencl/ramp/run.py and eventually by the main through the function run_opencl).
The original version Ramp-UA defines the path for the kernel_dir like this:

kernel_dir = os.path.join(opencl_dir, "ramp/kernels/")

# Load the OpenCL kernel programs
with open(os.path.join(kernel_dir, "ramp_ua.cl")) as f:
    program = cl.Program(ctx, f.read())
    program.build(options=[f"-I {kernel_dir}"])

In AZ's version, with the need to generalise the code, the path was initially rewritten taking the full path AND missed a "/" at the end (because the name is created automatically using os.path.join):

kernel_dir = os.path.join(Constants.Paths.PROJECT_FOLDER_ABSOLUTE_PATH,
                          Constants.Paths.SOURCE_FOLDER,
                          Constants.Paths.OPENCL.OPENCL_FOLDER,
                          Constants.Paths.OPENCL.OPENCL_SOURCE_FOLDER,
                          Constants.Paths.OPENCL.SOURCE.OPENCL_KERNELS_FOLDER)
print(f"Checking the kernel_dir \t{kernel_dir}\n")
# Load the OpenCL kernel programs
with open(os.path.join(kernel_dir,
                       Constants.Paths.OPENCL.SOURCE.KERNEL_FILE)) as f:
            program = cl.Program(ctx, f.read())
            program.build(options=[f"-I {kernel_dir}"])

The AZ edited model would run smoothly the opencl version without errors (even with the absolute path and a missing "/" at the end) until the files where moved to a specific folder, in order to use the Turing syncing service (ask AZ if you want details, she doesn't like white spaces in the naming!). Then the model would give error at the line 77 (AZ), so at the build function.
The error called is similar to this:
pyopencl.RuntimeError: clBuildProgram failed: invalid build options ...
(see Useful links) where it is easy to identify the issue because invalid build options lists only the of the part of the path that comes after the white space.
After a few investigations, noticed that this doesn't happen for the RAMP-UA original code, once moved to the same white-spaced-path folder.
The issue could be easily solved by not using folders whose paths has white spaces in the name (which is anyways a good habit), but we can't control what final users have on their computers, therefore it would be better to solve the problem throuroughly so to avoid running errors.

Actions undertaken to solve the issue
Created copies of both projects (UA and AZversion) in different ad-hoc folders, with spaces, no spaces, very loong names etc.
Debugged within the pyopencl code to understand where exactly the error was coming from.

Conclusions/Solution
Apparently the error comes from within OpenCL inner code (pyopencl works smoothly and accepts the path until the OpenCL build function is called).
Apparently OpenCL wants the kernel_dir with a final '/', so we need to declare it like that (this was not creating a problem when reading the kernel file at line , because there it was added by the os.path.join) though OpenCL builds wants as input variable kernel_dir and not the kernel file).
The final version of the code uses the relative path (not absolute) and a final "/". The relative path is declared within the model, where we can control the folders naming. Eventual white spaces in the user's folders naming are not entering the OpenCL core code, so they don't affect the model running.
Final version (current):

kernel_dir = os.path.join(Constants.Paths.SOURCE_FOLDER,
                          Constants.Paths.OPENCL.OPENCL_FOLDER,
                          Constants.Paths.OPENCL.OPENCL_SOURCE_FOLDER,
                          Constants.Paths.OPENCL.SOURCE.OPENCL_KERNELS_FOLDER)
kernel_dir = kernel_dir + "/"
print(f"Checking kernel_dir \t{kernel_dir}\n")
# Load the OpenCL kernel programs
with open(os.path.join(kernel_dir,
                       Constants.Paths.OPENCL.SOURCE.KERNEL_FILE)) as f:
    program = cl.Program(ctx, f.read())
    program.build(options=[f"-I {kernel_dir}"])

Useful links
https://stackoverflow.com/questions/22366517/pyopencl-runtimeerror-clbuildprogram-failed-invalid-build-options

Screenshots
None

* Another issue

Description of the problem

Actions undertaken to solve the issue

Conclusions/Solution

Useful links

Screenshots

Leave in-line comments…

TO_DO:

• first task
• second task
• last task