changed 6 years ago  
Linked with GitHub

HackyHour Potsdam 2019-02-06

When: February 6th, 2019 at 2:00pm CET
Where: Café Freundlich, Map
Info: Hacky Hour Potsdam Webseite

I have 8 CPU cores but my script uses only one

Participants

Exemple for Python multi-processing

import multiprocessing as mp
import numpy as np

###### EXEMPLE FUNCTIONS
###### creation of a simple function (sum of 3 digits)
def f(a,b,c):
    return a+b+c

###### creation of a wrap function (multiprocessing function don't like multi-argument functions)
def fwrap(argsabc):
    return f(*argsabc)

###### creation of a big list of random nunber (exemple specific) 
RandNum = list(np.random.rand(10000000,3))
print("END OF RANDOM NB GENERATION")

###### CREATION OF A POOL OBJECT
nbproc = 4 # number of parallel processing
Pool = mp.Pool(processes=nbproc)

###### PROCESSING ITSELF
### Solution 1
# (gives directly the results)
Results1 = Pool.map(fwrap, RandNum) 

### Solution 2
# (gives a list of objects, the results can be retrived with .get() method)
Results2_raw  = [Pool.apply_async(f, args=x) for x in RandNum]
Results2      = [e.get() for e in Results2]

For those still worried about threading and multiprocessing in python here
are some links from some older blog entries by Doug Hellmann
(author of the book "The Python Standard Library by Example"):

And here is the link to numba as well

And for OpenMP

Select a repo