python - Global vs local variable efficiency in multiple function call -


first caveat understand premature optimization bad. second caveat i'm new python.

i'm reading in many million data chunks. each chunk consists of 64 bits , held in numpy array. in order bit operations on numpy.uint64 type desired bit shift quantity must of same type:numpy.uint64.

this can accomplished either casting number or making variable.

number1 = numpy.uint64(80000) shift_amount = numpy.uint64(8) #option 1 number1 >> numpy.uint64(8) #option2 number1 >> shift_amount 

looping 10000 times , checking how long took. option2 wins out i'm assuming because overhead of creating numpy integer done once.

my current program calls function each chunk of data processes raw bits. function called millions of times , appends few different lists. assuming same idea , using globally defined values shift/bit operations 2 more loop conditions tested.

def using_global(number1):     global shift_amount     number1 >> shift_amount  def using_local(number1):     shift = np.uint64(54)     number1 >> shift  

looping these 10000 times function using global order of magnitude faster. question: bad practice have bunch(10+) global variables? https://wiki.python.org/moin/pythonspeed/performancetips states local variable faster. in instance found not case. main loop calls function each word in million of data words that's inefficient too.

python not made massive number operations. numpy instead is. put data junks in 1 numpy array. way faster using loops , single function calls:

values = numpy.ndarray(1000000, dtype=numpy.uint64) # fill in data values >>= 8 

if you're shift depends on highest nibble, example, nibble-values 0 15 have lookup table shift:

shift_by_nibble = numpy.array([8,16,24,30,34,60,50,40,44,48,52,56,62,4,12,20], dtype=numpy.uint8) values >>= shift_by_nibble[values>>60] 

Comments

Popular posts from this blog

python - No exponential form of the z-axis in matplotlib-3D-plots -

php - Best Light server (Linux + Web server + Database) for Raspberry Pi -

c# - "Newtonsoft.Json.JsonSerializationException unable to find constructor to use for types" error when deserializing class -