dataset. To make this easy, we will use transfer learning with ResNet50 trained on ImageNet weights. Please find the code from this post using . Run the commands in this section on your terminal.
JSON
export PROJECT_ID=<your-project-id>
gcloud config set project $PROJECT_ID
AI Platform Services
Please make sure to enable AI Platform Services for your GCP project by entering your project ID in for your new GCP project. A service account is an account used by an application or a . TensorFlow Cloud uses dataset for categorizing dog breeds. This is available as part of thePreprocessing
We will resize and batch the data. PYTHON
IMG_SIZE = 224
BATCH_SIZE = 64
BUFFER_SIZE = 2
size = (IMG_SIZE, IMG_SIZE)
ds_train = ds_train.map(lambda image, label: (tf.image.resize(image, size), label))
ds_test = ds_test.map(lambda image, label: (tf.image.resize(image, size), label))
def input_preprocess(image, label):
image = tf.keras.applications.resnet50.preprocess_input(image)
return image, labelConfigure the input pipeline for performance
Now we will configure the input pipeline for performance. Note that we are using parallel calls and prefetching so that I/O doesn’t become blocking while your model is training. You can learn more about configuring input pipelines for performance in this with weights trained on callback to save the model at various stages of training, callback to automatically determine the optimal number of epochs for training. PYTHON
MODEL_PATH = "resnet-dogs"
checkpoint_path = os.path.join("gs://", GCP_BUCKET, MODEL_PATH, "save_at_{epoch}")
tensorboard_path = os.path.join(
"gs://", GCP_BUCKET, "logs", datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
)
callbacks = [
tf.keras.callbacks.ModelCheckpoint(checkpoint_path),
tf.keras.callbacks.TensorBoard(log_dir=tensorboard_path, histogram_freq=1),
tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=3),
]Compile the model
PYTHON
optimizer = tf.keras.optimizers.Adam(learning_rate=1e-2)
model.compile(
optimizer=optimizer,
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=["accuracy"],
)Debug the model locally
We'll train the model in a local environment first in order to ensure that the code works properly before sending the job to GCP. We will usetfc.remote() to determine whether the code should be executed locally or on the cloud. Choosing a smaller number of epochs than intended for the full training job will help verify that the model is working properly without overloading your local machine. PYTHON
if tfc.remote():
epochs = 500
train_data = ds_train
test_data = ds_test
else:
epochs = 1
train_data = ds_train.take(5)
test_data = ds_test.take(5)
callbacks = None
model.fit(
train_data, epochs=epochs, callbacks=callbacks, validation_data=test_data, verbose=2
) PYTHON
if tfc.remote():
SAVE_PATH = os.path.join("gs://", GCP_BUCKET, MODEL_PATH)
model.save(SAVE_PATH)Model Training on Google Cloud
To train on GCP, populate the example code with your GCP project settings, then simply calltfc.run() from within your code. The API is simple with intelligent defaults for all the parameters. Again, we don’t need to worry about cloud specific tasks such as creating VM instances and distribution strategies when using TensorFlow Cloud. In order, the API will: - Make your python script/notebook cloud and distribution ready.
- Convert it into a docker image with required dependencies.
- Run the training job on a GCP cluster.
- Stream relevant logs and store checkpoints.
run() API provides significant flexibility for use, such as giving users the ability to specify custom cluster configuration, custom docker images. For a full list of parameters that can be used to call run(), see the TensorFlow Cloud Evaluate the model
After training, we can load the model that's been stored in our GCS bucket, and evaluate its performance. PYTHON
if tfc.remote():
model = tf.keras.models.load_model(SAVE_PATH)
model.evaluate(test_data)Next steps
This article introduced . As a next step, you can find this code example and many others on the TensorFlow Cloud repository. Vollständiger Original-Bericht
Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf blog.tensorflow.org.
SOCIAL SHARE CARD GENERATOR