python - Global vs local variable efficiency in multiple function call -
first caveat understand premature optimization bad. second caveat i'm new python.
i'm reading in many million data chunks. each chunk consists of 64 bits , held in numpy array. in order bit operations on numpy.uint64 type desired bit shift quantity must of same type:numpy.uint64.
this can accomplished either casting number or making variable.
number1 = numpy.uint64(80000) shift_amount = numpy.uint64(8) #option 1 number1 >> numpy.uint64(8) #option2 number1 >> shift_amount
looping 10000 times , checking how long took. option2 wins out i'm assuming because overhead of creating numpy integer done once.
my current program calls function each chunk of data processes raw bits. function called millions of times , appends few different lists. assuming same idea , using globally defined values shift/bit operations 2 more loop conditions tested.
def using_global(number1): global shift_amount number1 >> shift_amount def using_local(number1): shift = np.uint64(54) number1 >> shift
looping these 10000 times function using global order of magnitude faster. question: bad practice have bunch(10+) global variables? https://wiki.python.org/moin/pythonspeed/performancetips states local variable faster. in instance found not case. main loop calls function each word in million of data words that's inefficient too.
python not made massive number operations. numpy instead is. put data junks in 1 numpy array. way faster using loops , single function calls:
values = numpy.ndarray(1000000, dtype=numpy.uint64) # fill in data values >>= 8
if you're shift depends on highest nibble, example, nibble-values 0 15 have lookup table shift:
shift_by_nibble = numpy.array([8,16,24,30,34,60,50,40,44,48,52,56,62,4,12,20], dtype=numpy.uint8) values >>= shift_by_nibble[values>>60]
Comments
Post a Comment