, and TensorFlow Hub pages will link to the model implementations in the Model Garden.
will be maintained by the TensorFlow team to keep them up to date with the latest TensorFlow 2 APIs.
Model Garden can be easily installed using PIP (
pip install tf-models-nightly). TensorFlow 2 users can get started immediately with code examples to learn the best practice for training models on GPUs and TPUs.Many of the models in the Model Garden can be trained in a distributed fashion. In TensorFlow 2, you can distribute training workloads to single-host/multi-accelerator as well as multi-hosts/multi-accelerator configurations using the to find a list of the distribution strategies available for TensorFlow 2.
- - for multiple hosts each with multiple GPUs
- can classify the major object in the image into 1,000 object categories (e.g., car, soccer ball, table lamp, etc). The Model Garden provides an API. Here is an example to train a model with ImageNet data using two GPUs. The and ). For hyperparameter tuning, you may be interested in checking out environment variable at each GPU host to specify what tasks constitute a cluster, their addresses and each task's role in the cluster. Here is a typical example of
TF_CONFIGfor the first GPU host appointed as the chief worker:In this example, the “worker” part configures three GPU hosts (host1, host2, and host3) to run training using . You can use the same framework ().PYTHONos.environ["TF_CONFIG"] = json.dumps({
"cluster": {
"worker": ["host1:port", "host2:port", "host3:port"]
},
"task": {"type": "worker", "index": 0}})For those who want to have their own training loops rather than using TensorFlow's high-level API for building and training deep learning models, please check out (Bidirectional Encoder Representations from Transformers) and BERT variant models (e.g., ALBERT). Here we’ll demonstrate our best practices for training a TensorFlow 2 BERT model on GPUs and TPUs.PYTHON$ python3 classifier_trainer.py \
--mode=train_and_eval \
--model_type=resnet \
--dataset=imagenet \
--tpu=$TPU_NAME \
--model_dir=$MODEL_DIR \
--data_dir=$DATA_DIR \
--config_file=configs/examples/resnet/imagenet/tpu.yamlSentence and Sentence-pair Classification using BERT
Sentence and Sentence-pair classification task is to classify given a pair of sentences as paraphrases or not paraphrases. Here is an example to fine-tune the BERT-LARGE model using multiple GPUs on Google Cloud Platform. This task uses the Microsoft Research Paraphrase Corpus (MRPC) corpus that contains 5,801 pairs of sentences along with human annotations indicating whether each pair captures a paraphrase/semantic equivalence relationship.Similar to the TPU training for the ResNet model, users can easily switch to TPU for distributed training by changing the distribution strategy type to tpu with TPU information like the following example.PYTHONexport BERT_BASE_DIR=gs://cloud-tpu-checkpoints/bert/keras_bert/uncased_L-24_H-1024_A-16
export MODEL_DIR=gs://some_bucket/my_output_dir
export GLUE_DIR=gs://some_bucket/datasets
export TASK=MRPC
python3 run_classifier.py \
--mode='train_and_eval' \
--input_meta_data_path=${GLUE_DIR}/${TASK}_meta_data \
--train_data_path=${GLUE_DIR}/${TASK}_train.tf_record \
--eval_data_path=${GLUE_DIR}/${TASK}_eval.tf_record \
--bert_config_file=${BERT_BASE_DIR}/bert_config.json \
--init_checkpoint=${BERT_BASE_DIR}/bert_model.ckpt \
--train_batch_size=4 \
--eval_batch_size=4 \
--steps_per_loop=1 \
--learning_rate=2e-5 \
--num_train_epochs=3 \
--model_dir=${MODEL_DIR} \
--distribution_strategy=mirroredIf you want to use a pre-trained SavedModel provided by the to find the example codes we introduced in this article.PYTHON--distribution_strategy=tpu
--tpu=grpc://${TPU_IP_ADDRESS}:8470
In the coming months, we will provide more state-of-the-art canonical models and example code for building your own models. We would like to encourage AI researchers and developers to leverage the Model Garden when building their models.
We also welcome contributions to the repository to benefit the entire TensorFlow community. If you need any help, please reach out to us on GitHub.↗ Original-Artikel auf blog.tensorflow.org lesenVollständiger Original-BerichtAusführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org.
SOCIAL SHARE CARD GENERATOR