Check `get_folder_size_in_gb` function in compute worker code

[Didayolo]

This recursion feels weird in compute_worker.py:

def get_folder_size_in_gb(folder):
    if not os.path.exists(folder):
        return 0
    total_size = os.path.getsize(folder)
    for item in os.listdir(folder):
        path = os.path.join(folder, item)
        if os.path.isfile(path):
            total_size += os.path.getsize(path)
        elif os.path.isdir(path):
            total_size += get_folder_size_in_gb(path)
    return total_size / 1000 / 1000 / 1000  # GB: decimal system (1000^3)

Is this function correct?

[ihsaan-ullah]

There seems to be a problem with the unit becasue every recursive call returns in GB which is again converted to GB at the end

From GPT I got a simpler function with os.walk instead of recursion (TO BE TESTED)

def get_folder_size_in_gb(folder):
    total_size = 0
    for root, dirs, files in os.walk(folder):
        for f in files:
            fp = os.path.join(root, f)
            total_size += os.path.getsize(fp)
    
    return  total_size/ (1000 ** 3)