It didn’t take a long time to setup my dev environment. After the remote connection is done, which I have to do since the lab is running on some high-end CUDA machines, few well-labeled programs are gathered my own folder. A quick test is run against a sample video to separate it into frames, apply deep dreaming on the frames, then assemble the frames with sounds back to a dreamed video. The process takes few hours for ~1000 frames and the result is fine.

Iteration & Octave Interpolation

As mentioned in the previous post, iteration and octave are added by a new parameter pointing to a CSV file as follow:

frame,iteration,octave,layer
33,15,5,Layer
66,5,30,Layer
99,30,10,Layer

These values then can be interpolated linearly or by other models to generate a tuple list with the corresponding iteration, octave, layer (not implemented yet). Every two rows will be processed in one iteration which marks the start and the end of one keyframe section. One trick here is if the first keyframe is not starting from frame 1, the assumption is made that anything between frame 1 and this keyframe will have same iteration, octave, and layer. Same thing applies to the last keyframe and the last frame. The code sample is as follow:

# Must add one extra since frame start from 1 rather than 0
keyFrames = [(iterations, octaves)] * (int(totalFrame) + 1)
if key_frames is not None:
    with key_frames as csvfile:

        # Get first line to decide if the first frame is defined
        reader = csv.reader(csvfile, delimiter=',', quotechar='\'')
        reader = csv.reader(csvfile, delimiter=',', quotechar='\'')
        next(reader, None) # Skip the header
        firstLine = next(reader, None)
        if firstLine[0] != '1':
            previousRow = firstLine
            previousRow[0] = '1'
        else:
            previousRow = ''

        # Rewind the reader and read line by line
        csvfile.seek(0)
        next(reader, None) # Skip the header
        for row in reader:
            if previousRow != '':
                interpolate(keyFrames, previousRow, row, 'Linear')
            previousRow = row

        # Check last line and end interpolation properly
        if row[0] != str(totalFrame):
            lastRow = row[:]
            lastRow[0] = str(totalFrame)
            interpolate(keyFrames, row, lastRow, 'Linear')

After these sections are prepared, interpolation function is called. Currently, only the simple linear model is implemented where more advanced ones can be introduced in the future. One thing should keep in mind is that frame always starts from 1 rather than 0. The code sample is as follow:

def interpolate(
    keyFrames,
    previousRow,
    currentRow,
    method
):
    iterationFragment = (float(currentRow[1]) - float(previousRow[1])) /\
                        (float(currentRow[0]) - float(previousRow[0]))
    octaveFragment = (float(currentRow[2]) - float(previousRow[2])) /\
                     (float(currentRow[0]) - float(previousRow[0]))
    for i in range(0, int(currentRow[0]) - int(previousRow[0]) + 1):
        iteration = str(int(float(previousRow[1]) + i * iterationFragment))
        octave = str(int(float(previousRow[2]) + i * octaveFragment))
        keyFrames[int(previousRow[0]) + i] = (iteration, octave)

About Python

Apparently, I’m a Python hater 😀 One advice I’m always giving is do not start your programming career with Python. Reseason behind this is Python is such a different language compares to C-like style, fully adapt to Python will cause people difficult to move to other languages, i.e. I have friends tend to keep forgetting a variable need a type and how type casting works when moving from Python to others.

Other things are just pure basics, indentation rather than {}, don’t need to declare a variable before using it, no strict type. It literally dodged every single criterion where I think a good programming language should have. However, its “simplicity” and some well-supported science libraries made it gaining popularity in the research world. Nevertheless, one thing I learned from the past is if you are seriously thinking to work with one language style, you’d better choose the one pleasing you when writing.