silly question: how do I use this?

[mattfeury]

hi all!

first off, excuse my ignorance as i'm fairly new to ML, but have been playing with TF and Keras for the past few days and am feeling fairly comfortable. (However, Jupyter is new to me).

i'm very interested in using this to train a robust CNN that I can continuously use to identify vehicles as well as keep training as new data comes in. but i'm having trouble finding the entry points into the code. I've followed readme directions and spun up my docker instance, but it just opens what's essentially a file explorer in my browser (i assume this is jupyter). I'm also not sure what "Chips" are so the other documentation doc is a little confusing.

I would appreciate any sort of info you can provide. Not looking to be handheld, but any direction you can push me in would be very helpful. e.g. how would i go about training the network? what about feeding it a picture to identify? This seems like a robust and well-maintained repo, so i'd love to understand it.

thanks!

[d-grossman]

the processing chain roughly looks like:

1. video -> frames -> chips (best try at putting a car in a rectangle) -> now you have chips of cars
2. (chip1, chip2) -> siamese network for comparison -> (same car, same color, same body style)

All my work was in Keras; I did a writeup about training the siamese model: blog post

[d-grossman]

The various docker files:

Dockerfile.images2vecs maps turns a directory of chips into a file of vectors
Dockerfile.notebook jupyter notebook environment were we figured out all the dependencies
Dockerfile.rankDirectories compare two directories of chips of vehicles

[neuhargj]

@d-grossman do you have an end-to-end example (or script) for this? i'm using stanford's car dataset (http://ai.stanford.edu/~jkrause/cars/car_dataset.html), but i'm having trouble generating my siamese model from the data.
also, after reading your articles/posts on siamese networks, it looks like we want to run our inputs into two separate but identical subnetworks, and then the common top layers; does this translate roughly to doing the I2V work first and generating separate models for each set of images, and then running the siamese and ranker processes?

[d-grossman]

I would run all your images+augmentations through resnet50 with the top cut off to get vectors.

you would then be able to just use the vectors to iterate on your siamese top design.

When you have your siamese top design performing to your expectations then you can do end to end refinement.

You can start refinement code with:
https://github.com/Lab41/pelops/blob/master/pelops/analysis/siamese.ipynb

#cut top off
base_model = ResNet50(weights='imagenet', include_top=False)
input_left = Input(shape=(299, 299, 3))
input_right = Input(shape=(299, 299, 3))
processed_left = base_model(input_left)
processed_right = base_model(input_right)

#subtract (keep it, or not), if you dont keep the subtract you have 2x the num of features.
distance = Lambda(euclidean_distance,
                  output_shape=eucl_dist_output_shape)([processed_left, 
                                                        processed_right])

#add your layers 

#...

model = Model([input_left, input_right], distance)

rms = RMSprop()
model.compile(loss=contrastive_loss, optimizer=rms)