Tensorflow, Keras
# Resources
- http://playground.tensorflow.org/
- TensorFlow Hub
- https://www.tensorflow.org/hub
- Hundreds of trained, ready-to-deploy machine learning models in one place
- Keras Applications
- DL models that are made available alongside pre-trained weights. These models can be used for prediction, feature extraction, and fine-tuning
- Transfer Learning Guide: A Practical Tutorial With Examples for Images and Text in Keras
- https://cloud.google.com/blog/big-data/2016/07/understanding-neural-networks-with-tensorflow-playground
- https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc#.dg41ldof5
- https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0
- Tensorflow 2.0: models migration and new design
- http://planspace.org/20170404-how_not_to_program_the_tensorflow_graph/
- Understanding Tensorflow’s tensors shape: static and dynamic
- How to Write TensorFlow 2 Custom Loops
- Solving the TensorFlow Keras Model Loss Problem
- Building the Future of TensorFlow
# Mixed precision
# Distributed training
- https://missinglink.ai/guides/tensorflow/tensorflow-distributed-training-introduction-tutorials/
- TF.Distribute
- Numpy to tf.record
- https://www.tensorflow.org/tutorials/distribute/keras
- https://www.tensorflow.org/tutorials/distribute/custom_training
- https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/distribute/custom_training.ipynb#scrollTo=9iagoTBfijUz
- Train a Neural Network on multi-GPU with TensorFlow (Jordi Torres)
- Multinode example
- MultiWorkerMirroredStrategy:
- https://www.tensorflow.org/api_docs/python/tf/distribute/MultiWorkerMirroredStrategy (after TF v2)
- https://www.tensorflow.org/api_docs/python/tf/distribute/experimental/MultiWorkerMirroredStrategy
- https://blog.tensorflow.org/2020/12/whats-new-in-tensorflow-24.html (TF v2.4)
- https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras
- https://github.com/tensorflow/tensorflow/issues/36094
- https://www.tensorflow.org/api_docs/python/tf/distribute/cluster_resolver/SlurmClusterResolver
- https://lambdalabs.com/blog/tensorflow-2-0-tutorial-05-distributed-training-multi-node/
Data lazy loading:
- Tf.data:
- Tensorflow data netcdf, MATEX tensorflow (seems to be abandoned) :https://github.com/matex-org/matex/wiki/DataSet-Reader
# Tensorflow on M1/M2 Apple silicon
# Code
- #CODE Tensorflow (Google)
- #CODE
TF Compression
- TensorFlow Compression (TFC) contains data compression tools for TensorFlow
- #CODE
TF Similarity
- https://blog.tensorflow.org/2021/09/introducing-tensorflow-similarity.html
- Metric learning is different from traditional classification as it’s objective is different. The model learns to minimize the distance between similar examples and maximize the distance between dissimilar examples, in a supervised or self-supervised fashion
- #CODE
TF Probability
- Probabilistic reasoning and statistical analysis in TensorFlow
- https://www.tensorflow.org/probability
- #CODE
TF Decision Forests
- A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models in Keras
- https://blog.tensorflow.org/2021/05/introducing-tensorflow-decision-forests.html
- https://towardsdatascience.com/tensorflow-decision-forests-train-your-favorite-tree-based-models-using-keras-875d05a441f
- Decision forests in TF
- #CODE TF Datasets
- #CODE
TF Agents
- A reliable, scalable and easy to use TensorFlow library for Contextual Bandits and Reinforcement Learning
- #CODE
TF Recommenders
- Library for building recommender system models using TensorFlow.
- #CODE
TF Graphics
- Differentiable Graphics Layers for TensorFlow
- #CODE
TF Ranking
- Learning to Rank in TensorFlow
- #CODE
Tensorboard
- https://tensorboard.dev/
- https://www.tensorflow.org/tensorboard/get_started (use as a jupyterlab magic)
- #CODE Tensorflow.js
- #CODE TF On Spark
- #CODE Sonnet
- #CODE seq2seq
- #CODE Tensor2Tensor
- #CODE Graph Nets - Build Graph Nets in Tensorflow
- TF Graph Visualizer
- #CODE Tensorpack - It’s Yet Another TF high-level API, with speed, and flexibility built together
- #CODE Cleverhans - A library for benchmarking vulnerability to adversarial examples
# Keras
- #CODE
Keras
- Keras is a deep learning API written in Python, running on top of the machine learning platform Tensorflow
- http://keras.io/
- https://keras.io/getting_started/intro_to_keras_for_researchers/
- Modern Keras design patterns
- #CODE Keras-cv - Industry-strength Computer Vision workflows with Keras
- #CODE Keras-nlp - Modular Natural Language Processing workflows with Keras
- #CODE AutoKeras - Auto-Keras is an open source software library for automated machine learning (AutoML)
- #CODE Ktrain