PyTorch/Tensorflow

[ClashLuke]

Hello there,
I recently stumbled upon this repository and was interested in trying out your code. However, using single-threaded sklearn doesn't seem to be efficient to me, compared to using GPU-optimized PyTorch or TF.
Do you have any plans of moving to those frameworks, or would you accept a pullrequest implementing these?
Regards,
Luke

[alessiosavi]

Hi Clash,

Now i'm trying to understand which type of neural network suits better for recognize the 68 points extract from the face. So the work that you find here is only test/study purpose, for me and for everyone that need a basecode.

I'm currently changing the KNN in order to use a MLP classifier, that is obviosly more effifcient (in terms of precision) for this purpouse.

Of course, pull request are welcome!
I've played very little with PyTorch, and i think that Tensorflow will be a more preferable choiche.

[alessiosavi]

I've made some tests.
During the predict phase, the most time consuming process is the face encoding.

As you can see, encode two face cost ~3s on my hardware (GeForce 940MX).
It's because the jitter parameter used during the training/tuning phase have to be equal when make prediction, and i've choose 300 in order to increase the type of distortion made on the photos before training/predict.

Are you talking about tuning/training phase or predict?

---

Be sure to pull from master, i've migrated to MLP classifier that is more precise during prediction.

[ClashLuke]

First of all, thank you very much for taking your time to reply to this issue regarding training-optimization.

Second, you pointed out that the most time is consumed when encoding faces. This process could mostly be skipped, by using a couple keras features, such as the ImageDataGenerator.

Why does the jitter have to be equal during training and prediction? Isn't it normally used as a regularization technique, and therefore should be left out during inference, or am I thinking about something else here?

Lastly, I'd love to know if you'd be fine with a (backwards-compatible) switch to a CNN, so that we could compare the performance of Inception-v4 with an MLP.

[alessiosavi]

Hi Sir,

Thank you for the interest in the project!
You was completely right!

The problems related to the jitter parameter, was caused from the KNN that was not able to recognize the faces if trained using an high number of distortion. So far as that they does not correspond quite strictly (during the train/prediction phase), seems that the network is not very precise (i think that this have to be investigated in the archiecture of the KNN, but is out of scope).
With the MLP architecture (that have increase by an huge factor the confidence during the prediction), this little strange things is no more relevant, and we can use a different number of jitter during the training and predict phase.

Of course, the jitter parameter cause lot of time spent in image distortion (from documentation, jitter=300 -> 300x the time used), so use a different approach for create distortion will be a great performance improvement.
We have to understand the necessary parameters for ImageDataGenerator in order to preserve the quality of prediction (jitter make an average of the augmented data) and increase the speed of faces encodings.

From the architectural POV, we can play as much as we want with the code. So we can try lot of different type of network, using the same dataset for compare the result.

I expect that (with an higher amount of data), the CNN/RNN perform better. I suspect, instead, that with very few photos (and the majority of the ones of this dataset are lesser than 10), the MLP will perform slightly better.

During the change of the NN basecode (Classifer.py), it's important to maintain the possibility of recognize multiple faces in the same photo.

Before migrate to tensorflow, i think that are some work from my side in order to clean the code and standardize return function.