يوم مجتمع ML هو 9 نوفمبر! الانضمام إلينا للحصول على التحديثات من TensorFlow، JAX، وأكثر معرفة المزيد

تصنيف وضع الإنسان مع MoveNet و TensorFlow Lite

يعلمك هذا الكمبيوتر الدفتري كيفية تدريب نموذج تصنيف الوضعية باستخدام MoveNet و TensorFlow Lite. والنتيجة هي نموذج TensorFlow Lite جديد يقبل الإخراج من نموذج MoveNet كمدخلات ، ويخرج تصنيفًا للوضع ، مثل اسم وضعية اليوجا.

يتكون الإجراء في هذا الكمبيوتر الدفتري من 3 أجزاء:

  • الجزء 1: المعالجة المسبقة لبيانات تدريب تصنيف الوضعية في ملف CSV يحدد المعالم (نقاط مفاتيح الجسم) التي اكتشفها نموذج MoveNet ، جنبًا إلى جنب مع تسميات وضع الحقيقة على الأرض.
  • الجزء 2: بناء وتدريب نموذج تصنيف الوضعية الذي يأخذ إحداثيات المعالم من ملف CSV كمدخلات ، ويخرج التسميات المتوقعة.
  • الجزء 3: تحويل نموذج تصنيف الوضع إلى TFLite.

بشكل افتراضي ، يستخدم دفتر الملاحظات هذا مجموعة بيانات صور بها أوضاع يوغا معنونة ، لكننا قمنا أيضًا بتضمين قسم في الجزء 1 حيث يمكنك تحميل مجموعة بيانات الصور الخاصة بك للوضعيات.

عرض على TensorFlow.org تشغيل في Google Colab عرض المصدر على جيثب تحميل دفتر انظر نموذج TF Hub

تحضير

في هذا القسم ، ستقوم باستيراد المكتبات الضرورية وتعريف العديد من الوظائف للمعالجة المسبقة لصور التدريب في ملف CSV يحتوي على إحداثيات المعالم وتسميات الحقيقة الأساسية.

لا يوجد شيء يمكن ملاحظته هنا ، ولكن يمكنك توسيع خلايا الشفرة المخفية لمعرفة تنفيذ بعض الوظائف التي سنستدعيها لاحقًا.

إذا كنت تريد فقط إنشاء ملف CSV دون معرفة كل التفاصيل ، فما عليك سوى تشغيل هذا القسم والانتقال إلى الجزء الأول.

pip install -q opencv-python
import csv
import cv2
import itertools
import numpy as np
import pandas as pd
import os
import sys
import tempfile
import tqdm

from matplotlib import pyplot as plt
from matplotlib.collections import LineCollection

import tensorflow as tf
import tensorflow_hub as hub
from tensorflow import keras

from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

رمز لتشغيل تقدير الوضع باستخدام MoveNet

وظائف لتشغيل تقدير الوضع باستخدام MoveNet

Cloning into 'examples'...
remote: Enumerating objects: 19274, done.[K
remote: Counting objects: 100% (1094/1094), done.[K
remote: Compressing objects: 100% (606/606), done.[K
remote: Total 19274 (delta 455), reused 901 (delta 286), pack-reused 18180[K
Receiving objects: 100% (19274/19274), 30.59 MiB | 13.12 MiB/s, done.
Resolving deltas: 100% (10549/10549), done.

وظائف لتصور نتائج تقدير الوضع.

رمز لتحميل الصور واكتشاف معالم الوضع وحفظها في ملف CSV

(اختياري) مقتطف الشفرة لتجربة منطق تقدير وضع Movenet

--2021-10-07 13:04:44--  https://cdn.pixabay.com/photo/2017/03/03/17/30/yoga-2114512_960_720.jpg
Resolving cdn.pixabay.com (cdn.pixabay.com)... 104.18.21.183, 104.18.20.183, 2606:4700::6812:15b7, ...
Connecting to cdn.pixabay.com (cdn.pixabay.com)|104.18.21.183|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21743 (21K) [image/jpeg]
Saving to: ‘/tmp/image.jpeg’

/tmp/image.jpeg     100%[===================>]  21.23K  --.-KB/s    in 0s      

2021-10-07 13:04:45 (110 MB/s) - ‘/tmp/image.jpeg’ saved [21743/21743]

بي إن جي

الجزء 1: المعالجة المسبقة لصور الإدخال

لأن المدخلات لدينا المصنف تشكل هي معالم الإخراج من نموذج MoveNet، ونحن بحاجة لتوليد بيانات التدريب لدينا عن طريق تشغيل الصور وصفت من خلال MoveNet ومن ثم الاستيلاء على جميع البيانات التاريخية والعلامات الحقيقة على الارض في ملف CSV.

مجموعة البيانات التي قدمناها لهذا البرنامج التعليمي هي مجموعة بيانات وضعية اليوغا التي تم إنشاؤها بواسطة CG. يحتوي على صور لنماذج متعددة تم إنشاؤها بواسطة CG تقوم بـ 5 أوضاع يوغا مختلفة. الدليل ينقسم بالفعل في train مجموعة البيانات و test البيانات.

حتى في هذا القسم، سنقوم تحميل بيانات اليوغا وتشغيله من خلال MoveNet حتى نتمكن من التقاط كل المعالم في ملف CSV ... ومع ذلك، فإنه يأخذ حوالي 15 دقيقة لتغذية لدينا بيانات اليوغا لMoveNet وتوليد هذا الملف CSV . لذلك كبديل، يمكنك تحميل ملف CSV موجود مسبقا لمجموعة البيانات اليوغا من خلال وضع is_skip_step_1 المعلمة أدناه لصحيح. بهذه الطريقة ، ستتخطى هذه الخطوة وتنزيل بدلاً من ذلك ملف CSV نفسه الذي سيتم إنشاؤه في خطوة المعالجة المسبقة هذه.

من ناحية أخرى، إذا كنت ترغب في تدريب المصنف قفة مع بيانات الصورة الخاصة بك، تحتاج إلى تحميل الصور الخاصة بك وتشغيل هذه الخطوة تجهيزها (إجازة is_skip_step_1 خطأ) اتبع التعليمات التي تظهر أدناه لتحميل الخاص تشكل مجموعة البيانات الخاصة.

(اختياري) قم بتحميل مجموعة بيانات الوضع الخاصة بك

إذا كنت ترغب في تدريب مصنف الوضعية من خلال وضعياتك المحددة (يمكن أن تكون أي وضعيات ، وليس مجرد وضعيات يوغا) ، فاتبع الخطوات التالية:

  1. تعيين أعلاه use_custom_dataset خيار صحيح.

  2. قم بإعداد ملف أرشيف (ZIP أو TAR أو غير ذلك) يتضمن مجلدًا مع مجموعة بيانات الصور الخاصة بك. يجب أن يتضمن المجلد صورًا مرتبة لوضعياتك على النحو التالي.

    إذا كنت قد انشق بالفعل مجموعة البيانات الخاصة بك إلى تدريب واختبار مجموعات، ثم تعيين dataset_is_split إلى True. بمعنى ، يجب أن يتضمن مجلد الصور دليلي "القطار" و "الاختبار" مثل هذا:

    yoga_poses/
    |__ train/
        |__ downdog/
            |______ 00000128.jpg
            |______ ...
    |__ test/
        |__ downdog/
            |______ 00000181.jpg
            |______ ...
    

    أو، إذا ورقة العمل يتم تقسيم NOT بعد، ثم تعيين dataset_is_split إلى خطأ وسنقوم تقسيمها على أساس تقسيم جزء محدد. أي أن مجلد الصور التي تم تحميلها يجب أن يبدو كالتالي:

    yoga_poses/
    |__ downdog/
        |______ 00000128.jpg
        |______ 00000181.jpg
        |______ ...
    |__ goddess/
        |______ 00000243.jpg
        |______ 00000306.jpg
        |______ ...
    
  3. انقر فوق علامة التبويب الملفات على اليسار (رمز المجلد) ثم انقر فوق تحميل لتخزين جلسة (رمز الملف).

  4. حدد ملف الأرشيف الخاص بك وانتظر حتى ينتهي التحميل قبل المتابعة.

  5. قم بتحرير كتلة التعليمات البرمجية التالية لتحديد اسم ملف الأرشيف ودليل الصور. (بشكل افتراضي ، نتوقع ملف ZIP ، لذلك ستحتاج أيضًا إلى تعديل هذا الجزء إذا كان الأرشيف الخاص بك تنسيقًا آخر.)

  6. الآن قم بتشغيل بقية دفتر الملاحظات.

if use_custom_dataset:
  # ATTENTION:
  # You must edit these two lines to match your archive and images folder name:
  # !tar -xf YOUR_DATASET_ARCHIVE_NAME.tar
  !unzip -q YOUR_DATASET_ARCHIVE_NAME.zip
  dataset_in = 'YOUR_DATASET_DIR_NAME'

  # You can leave the rest alone:
  if not os.path.isdir(dataset_in):
    raise Exception("dataset_in is not a valid directory")
  if dataset_is_split:
    IMAGES_ROOT = dataset_in
  else:
    dataset_out = 'split_' + dataset_in
    split_into_train_test(dataset_in, dataset_out, test_split=0.2)
    IMAGES_ROOT = dataset_out

قم بتنزيل مجموعة بيانات اليوغا

if not is_skip_step_1 and not use_custom_dataset:
  !wget -O yoga_poses.zip http://download.tensorflow.org/data/pose_classification/yoga_poses.zip
  !unzip -q yoga_poses.zip -d yoga_cg
  IMAGES_ROOT = "yoga_cg"
--2021-10-07 13:04:47--  http://download.tensorflow.org/data/pose_classification/yoga_poses.zip
Resolving download.tensorflow.org (download.tensorflow.org)... 142.250.157.128, 2404:6800:4008:c13::80
Connecting to download.tensorflow.org (download.tensorflow.org)|142.250.157.128|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102517581 (98M) [application/zip]
Saving to: ‘yoga_poses.zip’

yoga_poses.zip      100%[===================>]  97.77M  81.3MB/s    in 1.2s    

2021-10-07 13:04:49 (81.3 MB/s) - ‘yoga_poses.zip’ saved [102517581/102517581]

المعالجة المسبقة ل TRAIN بيانات

if not is_skip_step_1:
  images_in_train_folder = os.path.join(IMAGES_ROOT, 'train')
  images_out_train_folder = 'poses_images_out_train'
  csvs_out_train_path = 'train_data.csv'

  preprocessor = MoveNetPreprocessor(
      images_in_folder=images_in_train_folder,
      images_out_folder=images_out_train_folder,
      csvs_out_path=csvs_out_train_path,
  )

  preprocessor.process(per_pose_class_limit=None)
Preprocessing chair
100%|██████████| 200/200 [00:57<00:00,  3.49it/s]
Preprocessing cobra
100%|██████████| 200/200 [00:53<00:00,  3.77it/s]
Preprocessing dog
100%|██████████| 200/200 [00:57<00:00,  3.46it/s]
Preprocessing tree
100%|██████████| 200/200 [00:57<00:00,  3.50it/s]
Preprocessing warrior
100%|██████████| 200/200 [00:58<00:00,  3.42it/s]
Skipped yoga_cg/train/cobra/girl1_cobra087.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra088.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra089.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra090.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra091.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra092.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra093.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra094.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra096.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra099.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra102.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra110.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra112.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra115.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra119.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra122.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra128.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra129.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra136.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra137.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl1_cobra140.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl2_cobra069.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl2_cobra078.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl2_cobra108.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl2_cobra117.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl2_cobra120.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra072.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra078.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra079.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra082.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra130.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra134.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/cobra/girl3_cobra138.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree086.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree087.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree088.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree089.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree090.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree094.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree096.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree099.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree100.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl1_tree101.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/tree/girl2_tree085.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/girl3_warrior090.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior134.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior135.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior138.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior143.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior145.jpg. No pose was confidentlly detected.
Skipped yoga_cg/train/warrior/guy1_warrior148.jpg. No pose was confidentlly detected.

المعالجة المسبقة ل TEST مجموعة البيانات

if not is_skip_step_1:
  images_in_test_folder = os.path.join(IMAGES_ROOT, 'test')
  images_out_test_folder = 'poses_images_out_test'
  csvs_out_test_path = 'test_data.csv'

  preprocessor = MoveNetPreprocessor(
      images_in_folder=images_in_test_folder,
      images_out_folder=images_out_test_folder,
      csvs_out_path=csvs_out_test_path,
  )

  preprocessor.process(per_pose_class_limit=None)
Preprocessing chair
100%|██████████| 84/84 [00:24<00:00,  3.39it/s]
Preprocessing cobra
100%|██████████| 116/116 [00:37<00:00,  3.05it/s]
Preprocessing dog
100%|██████████| 90/90 [00:27<00:00,  3.28it/s]
Preprocessing tree
100%|██████████| 96/96 [00:22<00:00,  4.26it/s]
Preprocessing warrior
100%|██████████| 109/109 [00:32<00:00,  3.37it/s]
Skipped yoga_cg/test/tree/guy3_tree089.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree090.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree091.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree092.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree093.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree094.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree095.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree096.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree097.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree098.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree099.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree100.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree101.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree102.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree103.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree104.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree105.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree106.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree107.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree108.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree109.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree110.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree111.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree112.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree113.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree114.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree115.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree116.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree117.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree118.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree119.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree120.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree121.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree122.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree123.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree124.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree125.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree126.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree127.jpg. No pose was confidentlly detected.
Skipped yoga_cg/test/tree/guy3_tree128.jpg. No pose was confidentlly detected.

الجزء 2: تدريب نموذج تصنيف الوضع الذي يأخذ إحداثيات المعالم كمدخلات ، وإخراج التسميات المتوقعة.

ستقوم ببناء نموذج TensorFlow يأخذ إحداثيات المعالم ويتنبأ بفئة الوضع التي يؤديها الشخص الموجود في صورة الإدخال. يتكون النموذج من نموذجين فرعيين:

  • يحسب النموذج الفرعي 1 تضمين الوضع (المعروف أيضًا باسم ناقل الميزة) من إحداثيات المعالم المكتشفة.
  • Submodel 2 يغذي تطرح التضمين من خلال عدة Dense طبقة التنبؤ الطبقة تشكل.

ستقوم بعد ذلك بتدريب النموذج بناءً على مجموعة البيانات التي تمت معالجتها مسبقًا في الجزء 1.

(اختياري) قم بتنزيل مجموعة البيانات المعالجة مسبقًا إذا لم تقم بتشغيل الجزء الأول

# Download the preprocessed CSV files which are the same as the output of step 1
if is_skip_step_1:
  !wget -O train_data.csv http://download.tensorflow.org/data/pose_classification/yoga_train_data.csv
  !wget -O test_data.csv http://download.tensorflow.org/data/pose_classification/yoga_test_data.csv

  csvs_out_train_path = 'train_data.csv'
  csvs_out_test_path = 'test_data.csv'
  is_skipped_step_1 = True

تحميل ملفات CSV preprocessed إلى TRAIN و TEST قواعد البيانات.

def load_pose_landmarks(csv_path):
  """Loads a CSV created by from the MoveNetPreprocessor.

  Returns:
    X: Detected landmark coordinates and scores of shape (N, 17 * 3)
    y: Ground truth labels of shape of shape (N, label_count)
    classes: The list of all class names found in the dataset
    dataframe: The CSV loaded as a Pandas dataframe features (X) and ground
      truth labels (y) to use later to train a pose classification model.
  """

  # Load the CSV file
  dataframe = pd.read_csv(csv_path)
  df_to_process = dataframe.copy()

  # Drop the file_name columns as you don't need it during training.
  df_to_process.drop(columns=['file_name'], inplace=True)

  # Extract the list of class names
  classes = df_to_process.pop('class_name').unique()

  # Extract the labels
  y = df_to_process.pop('class_no')

  # Convert the input features and labels into the correct format for training.
  X = df_to_process.astype('float64')
  y = keras.utils.to_categorical(y)

  return X, y, classes, dataframe

تحميل وتقسيم الأصلي TRAIN مجموعة البيانات إلى TRAIN (85٪ من البيانات) و VALIDATE (ال 15٪ الباقية).

# Load the train data
X, y, class_names, _ = load_pose_landmarks(csvs_out_train_path)

# Split training data (X, y) into (X_train, y_train) and (X_val, y_val)
X_train, X_val, y_train, y_val = train_test_split(X, y,
                                                  test_size=0.15)
# Load the test data
X_test, y_test, _, df_test = load_pose_landmarks(csvs_out_test_path)

حدد وظائف لتحويل معالم الوضع إلى تضمين وضع (يُعرف أيضًا باسم متجه الميزة) لتصنيف الوضعية

بعد ذلك ، قم بتحويل إحداثيات المعالم إلى متجه المعالم عن طريق:

  1. تحريك مركز الوضع إلى الأصل.
  2. تحجيم الوضعية بحيث يصبح حجم الوضعية 1
  3. تسطيح هذه الإحداثيات في ناقل سمة

ثم استخدم متجه الميزة هذا لتدريب مصنف وضعيات قائم على الشبكة العصبية.

def get_center_point(landmarks, left_name, right_name):
  """Calculates the center point of the two given landmarks."""

  left = tf.gather(landmarks, utils.KEYPOINT_DICT[left_name], axis=1)
  right = tf.gather(landmarks, utils.KEYPOINT_DICT[right_name], axis=1)
  center = left * 0.5 + right * 0.5
  return center


def get_pose_size(landmarks, torso_size_multiplier=2.5):
  """Calculates pose size.

  It is the maximum of two values:

    * Torso size multiplied by `torso_size_multiplier`
    * Maximum distance from pose center to any pose landmark
  """
  # Hips center
  hips_center = get_center_point(landmarks, "left_hip", "right_hip")

  # Shoulders center
  shoulders_center = get_center_point(landmarks, 
                                      "left_shoulder", "right_shoulder")

  # Torso size as the minimum body size
  torso_size = tf.linalg.norm(shoulders_center - hips_center)

  # Pose center
  pose_center_new = get_center_point(landmarks, "left_hip", "right_hip")
  pose_center_new = tf.expand_dims(pose_center_new, axis=1)
  # Broadcast the pose center to the same size as the landmark vector to
  # perform substraction
  pose_center_new = tf.broadcast_to(pose_center_new,
                                    [tf.size(landmarks) // (17*2), 17, 2])

  # Dist to pose center
  d = tf.gather(landmarks - pose_center_new, 0, axis=0,
                name="dist_to_pose_center")
  # Max dist to pose center
  max_dist = tf.reduce_max(tf.linalg.norm(d, axis=0))

  # Normalize scale
  pose_size = tf.maximum(torso_size * torso_size_multiplier, max_dist)

  return pose_size


def normalize_pose_landmarks(landmarks):
  """Normalizes the landmarks translation by moving the pose center to (0,0) and
  scaling it to a constant pose size.
  """
  # Move landmarks so that the pose center becomes (0,0)
  pose_center = get_center_point(landmarks, "left_hip", "right_hip")
  pose_center = tf.expand_dims(pose_center, axis=1)
  # Broadcast the pose center to the same size as the landmark vector to perform
  # substraction
  pose_center = tf.broadcast_to(pose_center, 
                                [tf.size(landmarks) // (17*2), 17, 2])
  landmarks = landmarks - pose_center

  # Scale the landmarks to a constant pose size
  pose_size = get_pose_size(landmarks)
  landmarks /= pose_size

  return landmarks


def landmarks_to_embedding(landmarks_and_scores):
  """Converts the input landmarks into a pose embedding."""
  # Reshape the flat input into a matrix with shape=(17, 3)
  reshaped_inputs = keras.layers.Reshape((17, 3))(landmarks_and_scores)

  # Normalize landmarks 2D
  landmarks = normalize_pose_landmarks(reshaped_inputs[:, :, :2])

  # Flatten the normalized landmark coordinates into a vector
  embedding = keras.layers.Flatten()(landmarks)

  return embedding

تحديد نموذج Keras لتصنيف الوضعية

يأخذ نموذج Keras الخاص بنا معالم الوضع المكتشفة ، ثم يحسب تضمين الوضع ويتنبأ بفئة الوضع.

# Define the model
inputs = tf.keras.Input(shape=(51))
embedding = landmarks_to_embedding(inputs)

layer = keras.layers.Dense(128, activation=tf.nn.relu6)(embedding)
layer = keras.layers.Dropout(0.5)(layer)
layer = keras.layers.Dense(64, activation=tf.nn.relu6)(layer)
layer = keras.layers.Dropout(0.5)(layer)
outputs = keras.layers.Dense(5, activation="softmax")(layer)

model = keras.Model(inputs, outputs)
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 51)]         0                                            
__________________________________________________________________________________________________
reshape (Reshape)               (None, 17, 3)        0           input_1[0][0]                    
__________________________________________________________________________________________________
tf.__operators__.getitem (Slici (None, 17, 2)        0           reshape[0][0]                    
__________________________________________________________________________________________________
tf.compat.v1.gather (TFOpLambda (None, 2)            0           tf.__operators__.getitem[0][0]   
__________________________________________________________________________________________________
tf.compat.v1.gather_1 (TFOpLamb (None, 2)            0           tf.__operators__.getitem[0][0]   
__________________________________________________________________________________________________
tf.math.multiply (TFOpLambda)   (None, 2)            0           tf.compat.v1.gather[0][0]        
__________________________________________________________________________________________________
tf.math.multiply_1 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_1[0][0]      
__________________________________________________________________________________________________
tf.__operators__.add (TFOpLambd (None, 2)            0           tf.math.multiply[0][0]           
                                                                 tf.math.multiply_1[0][0]         
__________________________________________________________________________________________________
tf.compat.v1.size (TFOpLambda)  ()                   0           tf.__operators__.getitem[0][0]   
__________________________________________________________________________________________________
tf.expand_dims (TFOpLambda)     (None, 1, 2)         0           tf.__operators__.add[0][0]       
__________________________________________________________________________________________________
tf.compat.v1.floor_div (TFOpLam ()                   0           tf.compat.v1.size[0][0]          
__________________________________________________________________________________________________
tf.broadcast_to (TFOpLambda)    (None, 17, 2)        0           tf.expand_dims[0][0]             
                                                                 tf.compat.v1.floor_div[0][0]     
__________________________________________________________________________________________________
tf.math.subtract (TFOpLambda)   (None, 17, 2)        0           tf.__operators__.getitem[0][0]   
                                                                 tf.broadcast_to[0][0]            
__________________________________________________________________________________________________
tf.compat.v1.gather_6 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.compat.v1.gather_7 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.math.multiply_6 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_6[0][0]      
__________________________________________________________________________________________________
tf.math.multiply_7 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_7[0][0]      
__________________________________________________________________________________________________
tf.__operators__.add_3 (TFOpLam (None, 2)            0           tf.math.multiply_6[0][0]         
                                                                 tf.math.multiply_7[0][0]         
__________________________________________________________________________________________________
tf.compat.v1.size_1 (TFOpLambda ()                   0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.compat.v1.gather_4 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.compat.v1.gather_5 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.compat.v1.gather_2 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.compat.v1.gather_3 (TFOpLamb (None, 2)            0           tf.math.subtract[0][0]           
__________________________________________________________________________________________________
tf.expand_dims_1 (TFOpLambda)   (None, 1, 2)         0           tf.__operators__.add_3[0][0]     
__________________________________________________________________________________________________
tf.compat.v1.floor_div_1 (TFOpL ()                   0           tf.compat.v1.size_1[0][0]        
__________________________________________________________________________________________________
tf.math.multiply_4 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_4[0][0]      
__________________________________________________________________________________________________
tf.math.multiply_5 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_5[0][0]      
__________________________________________________________________________________________________
tf.math.multiply_2 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_2[0][0]      
__________________________________________________________________________________________________
tf.math.multiply_3 (TFOpLambda) (None, 2)            0           tf.compat.v1.gather_3[0][0]      
__________________________________________________________________________________________________
tf.broadcast_to_1 (TFOpLambda)  (None, 17, 2)        0           tf.expand_dims_1[0][0]           
                                                                 tf.compat.v1.floor_div_1[0][0]   
__________________________________________________________________________________________________
tf.__operators__.add_2 (TFOpLam (None, 2)            0           tf.math.multiply_4[0][0]         
                                                                 tf.math.multiply_5[0][0]         
__________________________________________________________________________________________________
tf.__operators__.add_1 (TFOpLam (None, 2)            0           tf.math.multiply_2[0][0]         
                                                                 tf.math.multiply_3[0][0]         
__________________________________________________________________________________________________
tf.math.subtract_2 (TFOpLambda) (None, 17, 2)        0           tf.math.subtract[0][0]           
                                                                 tf.broadcast_to_1[0][0]          
__________________________________________________________________________________________________
tf.math.subtract_1 (TFOpLambda) (None, 2)            0           tf.__operators__.add_2[0][0]     
                                                                 tf.__operators__.add_1[0][0]     
__________________________________________________________________________________________________
tf.compat.v1.gather_8 (TFOpLamb (17, 2)              0           tf.math.subtract_2[0][0]         
__________________________________________________________________________________________________
tf.compat.v1.norm (TFOpLambda)  ()                   0           tf.math.subtract_1[0][0]         
__________________________________________________________________________________________________
tf.compat.v1.norm_1 (TFOpLambda (2,)                 0           tf.compat.v1.gather_8[0][0]      
__________________________________________________________________________________________________
tf.math.multiply_8 (TFOpLambda) ()                   0           tf.compat.v1.norm[0][0]          
__________________________________________________________________________________________________
tf.math.reduce_max (TFOpLambda) ()                   0           tf.compat.v1.norm_1[0][0]        
__________________________________________________________________________________________________
tf.math.maximum (TFOpLambda)    ()                   0           tf.math.multiply_8[0][0]         
                                                                 tf.math.reduce_max[0][0]         
__________________________________________________________________________________________________
tf.math.truediv (TFOpLambda)    (None, 17, 2)        0           tf.math.subtract[0][0]           
                                                                 tf.math.maximum[0][0]            
__________________________________________________________________________________________________
flatten (Flatten)               (None, 34)           0           tf.math.truediv[0][0]            
__________________________________________________________________________________________________
dense (Dense)                   (None, 128)          4480        flatten[0][0]                    
__________________________________________________________________________________________________
dropout (Dropout)               (None, 128)          0           dense[0][0]                      
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 64)           8256        dropout[0][0]                    
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 64)           0           dense_1[0][0]                    
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 5)            325         dropout_1[0][0]                  
==================================================================================================
Total params: 13,061
Trainable params: 13,061
Non-trainable params: 0
__________________________________________________________________________________________________
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)

# Add a checkpoint callback to store the checkpoint that has the highest
# validation accuracy.
checkpoint_path = "weights.best.hdf5"
checkpoint = keras.callbacks.ModelCheckpoint(checkpoint_path,
                             monitor='val_accuracy',
                             verbose=1,
                             save_best_only=True,
                             mode='max')
earlystopping = keras.callbacks.EarlyStopping(monitor='val_accuracy', 
                                              patience=20)

# Start training
history = model.fit(X_train, y_train,
                    epochs=200,
                    batch_size=16,
                    validation_data=(X_val, y_val),
                    callbacks=[checkpoint, earlystopping])
Epoch 1/200
51/51 [==============================] - 1s 8ms/step - loss: 1.4885 - accuracy: 0.4268 - val_loss: 1.3159 - val_accuracy: 0.5175

Epoch 00001: val_accuracy improved from -inf to 0.51748, saving model to weights.best.hdf5
Epoch 2/200
51/51 [==============================] - 0s 3ms/step - loss: 1.1835 - accuracy: 0.5248 - val_loss: 1.0428 - val_accuracy: 0.5944

Epoch 00002: val_accuracy improved from 0.51748 to 0.59441, saving model to weights.best.hdf5
Epoch 3/200
51/51 [==============================] - 0s 3ms/step - loss: 0.9820 - accuracy: 0.5720 - val_loss: 0.8578 - val_accuracy: 0.6923

Epoch 00003: val_accuracy improved from 0.59441 to 0.69231, saving model to weights.best.hdf5
Epoch 4/200
51/51 [==============================] - 0s 3ms/step - loss: 0.8697 - accuracy: 0.6476 - val_loss: 0.7241 - val_accuracy: 0.7762

Epoch 00004: val_accuracy improved from 0.69231 to 0.77622, saving model to weights.best.hdf5
Epoch 5/200
51/51 [==============================] - 0s 3ms/step - loss: 0.7163 - accuracy: 0.7543 - val_loss: 0.6160 - val_accuracy: 0.7762

Epoch 00005: val_accuracy did not improve from 0.77622
Epoch 6/200
51/51 [==============================] - 0s 3ms/step - loss: 0.6695 - accuracy: 0.7382 - val_loss: 0.5000 - val_accuracy: 0.8601

Epoch 00006: val_accuracy improved from 0.77622 to 0.86014, saving model to weights.best.hdf5
Epoch 7/200
51/51 [==============================] - 0s 3ms/step - loss: 0.6053 - accuracy: 0.7829 - val_loss: 0.4317 - val_accuracy: 0.8951

Epoch 00007: val_accuracy improved from 0.86014 to 0.89510, saving model to weights.best.hdf5
Epoch 8/200
51/51 [==============================] - 0s 3ms/step - loss: 0.4981 - accuracy: 0.8325 - val_loss: 0.3589 - val_accuracy: 0.8951

Epoch 00008: val_accuracy did not improve from 0.89510
Epoch 9/200
51/51 [==============================] - 0s 3ms/step - loss: 0.4573 - accuracy: 0.8474 - val_loss: 0.3141 - val_accuracy: 0.9371

Epoch 00009: val_accuracy improved from 0.89510 to 0.93706, saving model to weights.best.hdf5
Epoch 10/200
51/51 [==============================] - 0s 3ms/step - loss: 0.4152 - accuracy: 0.8697 - val_loss: 0.2566 - val_accuracy: 0.9441

Epoch 00010: val_accuracy improved from 0.93706 to 0.94406, saving model to weights.best.hdf5
Epoch 11/200
51/51 [==============================] - 0s 3ms/step - loss: 0.3706 - accuracy: 0.8772 - val_loss: 0.2344 - val_accuracy: 0.9441

Epoch 00011: val_accuracy did not improve from 0.94406
Epoch 12/200
51/51 [==============================] - 0s 3ms/step - loss: 0.3246 - accuracy: 0.8945 - val_loss: 0.2130 - val_accuracy: 0.9371

Epoch 00012: val_accuracy did not improve from 0.94406
Epoch 13/200
51/51 [==============================] - 0s 3ms/step - loss: 0.3243 - accuracy: 0.8921 - val_loss: 0.1833 - val_accuracy: 0.9510

Epoch 00013: val_accuracy improved from 0.94406 to 0.95105, saving model to weights.best.hdf5
Epoch 14/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2976 - accuracy: 0.9082 - val_loss: 0.1661 - val_accuracy: 0.9510

Epoch 00014: val_accuracy did not improve from 0.95105
Epoch 15/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2685 - accuracy: 0.9069 - val_loss: 0.1482 - val_accuracy: 0.9580

Epoch 00015: val_accuracy improved from 0.95105 to 0.95804, saving model to weights.best.hdf5
Epoch 16/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2666 - accuracy: 0.9119 - val_loss: 0.1450 - val_accuracy: 0.9580

Epoch 00016: val_accuracy did not improve from 0.95804
Epoch 17/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2249 - accuracy: 0.9218 - val_loss: 0.1289 - val_accuracy: 0.9650

Epoch 00017: val_accuracy improved from 0.95804 to 0.96503, saving model to weights.best.hdf5
Epoch 18/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2291 - accuracy: 0.9392 - val_loss: 0.1205 - val_accuracy: 0.9580

Epoch 00018: val_accuracy did not improve from 0.96503
Epoch 19/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2205 - accuracy: 0.9342 - val_loss: 0.1106 - val_accuracy: 0.9650

Epoch 00019: val_accuracy did not improve from 0.96503
Epoch 20/200
51/51 [==============================] - 0s 3ms/step - loss: 0.2008 - accuracy: 0.9404 - val_loss: 0.1161 - val_accuracy: 0.9580

Epoch 00020: val_accuracy did not improve from 0.96503
Epoch 21/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1921 - accuracy: 0.9380 - val_loss: 0.0999 - val_accuracy: 0.9650

Epoch 00021: val_accuracy did not improve from 0.96503
Epoch 22/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1875 - accuracy: 0.9467 - val_loss: 0.0927 - val_accuracy: 0.9650

Epoch 00022: val_accuracy did not improve from 0.96503
Epoch 23/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1910 - accuracy: 0.9392 - val_loss: 0.0971 - val_accuracy: 0.9580

Epoch 00023: val_accuracy did not improve from 0.96503
Epoch 24/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1540 - accuracy: 0.9529 - val_loss: 0.0897 - val_accuracy: 0.9650

Epoch 00024: val_accuracy did not improve from 0.96503
Epoch 25/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1585 - accuracy: 0.9516 - val_loss: 0.0812 - val_accuracy: 0.9650

Epoch 00025: val_accuracy did not improve from 0.96503
Epoch 26/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1538 - accuracy: 0.9504 - val_loss: 0.0774 - val_accuracy: 0.9720

Epoch 00026: val_accuracy improved from 0.96503 to 0.97203, saving model to weights.best.hdf5
Epoch 27/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1420 - accuracy: 0.9603 - val_loss: 0.0786 - val_accuracy: 0.9650

Epoch 00027: val_accuracy did not improve from 0.97203
Epoch 28/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1601 - accuracy: 0.9529 - val_loss: 0.0714 - val_accuracy: 0.9720

Epoch 00028: val_accuracy did not improve from 0.97203
Epoch 29/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1319 - accuracy: 0.9578 - val_loss: 0.0661 - val_accuracy: 0.9720

Epoch 00029: val_accuracy did not improve from 0.97203
Epoch 30/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1302 - accuracy: 0.9591 - val_loss: 0.0617 - val_accuracy: 0.9790

Epoch 00030: val_accuracy improved from 0.97203 to 0.97902, saving model to weights.best.hdf5
Epoch 31/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1239 - accuracy: 0.9653 - val_loss: 0.0673 - val_accuracy: 0.9720

Epoch 00031: val_accuracy did not improve from 0.97902
Epoch 32/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1269 - accuracy: 0.9553 - val_loss: 0.0550 - val_accuracy: 0.9790

Epoch 00032: val_accuracy did not improve from 0.97902
Epoch 33/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1317 - accuracy: 0.9553 - val_loss: 0.0517 - val_accuracy: 0.9790

Epoch 00033: val_accuracy did not improve from 0.97902
Epoch 34/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1155 - accuracy: 0.9640 - val_loss: 0.0554 - val_accuracy: 0.9720

Epoch 00034: val_accuracy did not improve from 0.97902
Epoch 35/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1055 - accuracy: 0.9727 - val_loss: 0.0564 - val_accuracy: 0.9720

Epoch 00035: val_accuracy did not improve from 0.97902
Epoch 36/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1084 - accuracy: 0.9653 - val_loss: 0.0504 - val_accuracy: 0.9790

Epoch 00036: val_accuracy did not improve from 0.97902
Epoch 37/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0862 - accuracy: 0.9752 - val_loss: 0.0452 - val_accuracy: 0.9860

Epoch 00037: val_accuracy improved from 0.97902 to 0.98601, saving model to weights.best.hdf5
Epoch 38/200
51/51 [==============================] - 0s 3ms/step - loss: 0.1096 - accuracy: 0.9665 - val_loss: 0.0478 - val_accuracy: 0.9860

Epoch 00038: val_accuracy did not improve from 0.98601
Epoch 39/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0990 - accuracy: 0.9690 - val_loss: 0.0395 - val_accuracy: 0.9930

Epoch 00039: val_accuracy improved from 0.98601 to 0.99301, saving model to weights.best.hdf5
Epoch 40/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0966 - accuracy: 0.9727 - val_loss: 0.0404 - val_accuracy: 0.9860

Epoch 00040: val_accuracy did not improve from 0.99301
Epoch 41/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0906 - accuracy: 0.9764 - val_loss: 0.0417 - val_accuracy: 0.9790

Epoch 00041: val_accuracy did not improve from 0.99301
Epoch 42/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0826 - accuracy: 0.9752 - val_loss: 0.0391 - val_accuracy: 0.9860

Epoch 00042: val_accuracy did not improve from 0.99301
Epoch 43/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0797 - accuracy: 0.9764 - val_loss: 0.0423 - val_accuracy: 0.9790

Epoch 00043: val_accuracy did not improve from 0.99301
Epoch 44/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0802 - accuracy: 0.9789 - val_loss: 0.0382 - val_accuracy: 0.9860

Epoch 00044: val_accuracy did not improve from 0.99301
Epoch 45/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0992 - accuracy: 0.9677 - val_loss: 0.0360 - val_accuracy: 0.9860

Epoch 00045: val_accuracy did not improve from 0.99301
Epoch 46/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0821 - accuracy: 0.9752 - val_loss: 0.0344 - val_accuracy: 0.9860

Epoch 00046: val_accuracy did not improve from 0.99301
Epoch 47/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0746 - accuracy: 0.9789 - val_loss: 0.0394 - val_accuracy: 0.9860

Epoch 00047: val_accuracy did not improve from 0.99301
Epoch 48/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0738 - accuracy: 0.9864 - val_loss: 0.0327 - val_accuracy: 0.9930

Epoch 00048: val_accuracy did not improve from 0.99301
Epoch 49/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0827 - accuracy: 0.9715 - val_loss: 0.0302 - val_accuracy: 0.9930

Epoch 00049: val_accuracy did not improve from 0.99301
Epoch 50/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0818 - accuracy: 0.9727 - val_loss: 0.0319 - val_accuracy: 0.9930

Epoch 00050: val_accuracy did not improve from 0.99301
Epoch 51/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0804 - accuracy: 0.9801 - val_loss: 0.0239 - val_accuracy: 0.9930

Epoch 00051: val_accuracy did not improve from 0.99301
Epoch 52/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0748 - accuracy: 0.9727 - val_loss: 0.0300 - val_accuracy: 0.9930

Epoch 00052: val_accuracy did not improve from 0.99301
Epoch 53/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0694 - accuracy: 0.9826 - val_loss: 0.0234 - val_accuracy: 0.9930

Epoch 00053: val_accuracy did not improve from 0.99301
Epoch 54/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0739 - accuracy: 0.9777 - val_loss: 0.0297 - val_accuracy: 0.9930

Epoch 00054: val_accuracy did not improve from 0.99301
Epoch 55/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0671 - accuracy: 0.9826 - val_loss: 0.0248 - val_accuracy: 0.9930

Epoch 00055: val_accuracy did not improve from 0.99301
Epoch 56/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0764 - accuracy: 0.9702 - val_loss: 0.0268 - val_accuracy: 0.9930

Epoch 00056: val_accuracy did not improve from 0.99301
Epoch 57/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0676 - accuracy: 0.9777 - val_loss: 0.0226 - val_accuracy: 0.9930

Epoch 00057: val_accuracy did not improve from 0.99301
Epoch 58/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0504 - accuracy: 0.9851 - val_loss: 0.0285 - val_accuracy: 0.9930

Epoch 00058: val_accuracy did not improve from 0.99301
Epoch 59/200
51/51 [==============================] - 0s 3ms/step - loss: 0.0520 - accuracy: 0.9864 - val_loss: 0.0285 - val_accuracy: 0.9930

Epoch 00059: val_accuracy did not improve from 0.99301
# Visualize the training history to see whether you're overfitting.
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['TRAIN', 'VAL'], loc='lower right')
plt.show()

بي إن جي

# Evaluate the model using the TEST dataset
loss, accuracy = model.evaluate(X_test, y_test)
15/15 [==============================] - 0s 2ms/step - loss: 0.0217 - accuracy: 0.9934

ارسم مصفوفة الارتباك لفهم أداء النموذج بشكل أفضل

def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
  """Plots the confusion matrix."""
  if normalize:
    cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
    print("Normalized confusion matrix")
  else:
    print('Confusion matrix, without normalization')

  plt.imshow(cm, interpolation='nearest', cmap=cmap)
  plt.title(title)
  plt.colorbar()
  tick_marks = np.arange(len(classes))
  plt.xticks(tick_marks, classes, rotation=55)
  plt.yticks(tick_marks, classes)
  fmt = '.2f' if normalize else 'd'
  thresh = cm.max() / 2.
  for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
    plt.text(j, i, format(cm[i, j], fmt),
              horizontalalignment="center",
              color="white" if cm[i, j] > thresh else "black")

  plt.ylabel('True label')
  plt.xlabel('Predicted label')
  plt.tight_layout()

# Classify pose in the TEST dataset using the trained model
y_pred = model.predict(X_test)

# Convert the prediction result to class name
y_pred_label = [class_names[i] for i in np.argmax(y_pred, axis=1)]
y_true_label = [class_names[i] for i in np.argmax(y_test, axis=1)]

# Plot the confusion matrix
cm = confusion_matrix(np.argmax(y_test, axis=1), np.argmax(y_pred, axis=1))
plot_confusion_matrix(cm,
                      class_names,
                      title ='Confusion Matrix of Pose Classification Model')

# Print the classification report
print('\nClassification Report:\n', classification_report(y_true_label,
                                                          y_pred_label))
Confusion matrix, without normalization

Classification Report:
               precision    recall  f1-score   support

       chair       1.00      1.00      1.00        84
       cobra       0.97      1.00      0.99       116
         dog       1.00      1.00      1.00        90
        tree       1.00      1.00      1.00        56
     warrior       1.00      0.97      0.99       109

    accuracy                           0.99       455
   macro avg       0.99      0.99      0.99       455
weighted avg       0.99      0.99      0.99       455

بي إن جي

(اختياري) تحقق من التنبؤات غير الصحيحة

يمكنك أن تبحث في يطرح من TEST مجموعة البيانات التي تم التنبؤ بشكل غير صحيح لمعرفة ما إذا كانت دقة نموذج يمكن تحسينها.

if is_skip_step_1:
  print('ERROR: You must have run step 1 to run this cell.')
else:
  # If step 1 was skipped, skip this step.
  IMAGE_PER_ROW = 3
  MAX_NO_OF_IMAGE_TO_PLOT = 30

  # Extract the list of incorrectly predicted poses
  false_predict = [id_in_df for id_in_df in range(len(y_test)) \
                  if y_pred_label[id_in_df] != y_true_label[id_in_df]]
  if len(false_predict) > MAX_NO_OF_IMAGE_TO_PLOT:
    false_predict = false_predict[:MAX_NO_OF_IMAGE_TO_PLOT]

  # Plot the incorrectly predicted images
  row_count = len(false_predict) // IMAGE_PER_ROW + 1
  fig = plt.figure(figsize=(10 * IMAGE_PER_ROW, 10 * row_count))
  for i, id_in_df in enumerate(false_predict):
    ax = fig.add_subplot(row_count, IMAGE_PER_ROW, i + 1)
    image_path = os.path.join(images_out_test_folder,
                              df_test.iloc[id_in_df]['file_name'])

    image = cv2.imread(image_path)
    plt.title("Predict: %s; Actual: %s"
              % (y_pred_label[id_in_df], y_true_label[id_in_df]))
    plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
  plt.show()

بي إن جي

الجزء 3: تحويل نموذج تصنيف الوضع إلى TensorFlow Lite

ستقوم بتحويل نموذج تصنيف وضع Keras إلى تنسيق TensorFlow Lite بحيث يمكنك نشره على تطبيقات الجوال ومتصفحات الويب وأجهزة إنترنت الأشياء. عند تحويل هذا النموذج، عليك تطبيق النطاق الديناميكي تكميم للحد من تصنيف تشكل TensorFlow لايت حجم النموذج من قبل حوالي 4 مرات مع فقدان دقة تافهة.

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_model = converter.convert()

print('Model size: %dKB' % (len(tflite_model) / 1024))

with open('pose_classifier.tflite', 'wb') as f:
  f.write(tflite_model)
2021-10-07 13:12:14.380528: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: /tmp/tmpt4btgxr9/assets
Model size: 26KB
2021-10-07 13:12:15.585943: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:351] Ignored output_format.
2021-10-07 13:12:15.585983: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:354] Ignored drop_control_dependency.

ثم ستكتب ملف التسمية الذي يحتوي على تعيين من فهارس الفئات إلى أسماء الفئات التي يمكن قراءتها.

with open('pose_labels.txt', 'w') as f:
  f.write('\n'.join(class_names))

نظرًا لأنك قمت بتطبيق التكميم لتقليل حجم النموذج ، فلنقم بتقييم نموذج TFLite الكمي للتحقق مما إذا كان انخفاض الدقة مقبولاً.

def evaluate_model(interpreter, X, y_true):
  """Evaluates the given TFLite model and return its accuracy."""
  input_index = interpreter.get_input_details()[0]["index"]
  output_index = interpreter.get_output_details()[0]["index"]

  # Run predictions on all given poses.
  y_pred = []
  for i in range(len(y_true)):
    # Pre-processing: add batch dimension and convert to float32 to match with
    # the model's input data format.
    test_image = X[i: i + 1].astype('float32')
    interpreter.set_tensor(input_index, test_image)

    # Run inference.
    interpreter.invoke()

    # Post-processing: remove batch dimension and find the class with highest
    # probability.
    output = interpreter.tensor(output_index)
    predicted_label = np.argmax(output()[0])
    y_pred.append(predicted_label)

  # Compare prediction results with ground truth labels to calculate accuracy.
  y_pred = keras.utils.to_categorical(y_pred)
  return accuracy_score(y_true, y_pred)

# Evaluate the accuracy of the converted TFLite model
classifier_interpreter = tf.lite.Interpreter(model_content=tflite_model)
classifier_interpreter.allocate_tensors()
print('Accuracy of TFLite model: %s' %
      evaluate_model(classifier_interpreter, X_test, y_test))
Accuracy of TFLite model: 0.9978021978021978

الآن يمكنك تحميل نموذج TFLite ( pose_classifier.tflite ) وملف تسمية ( pose_labels.txt ) ليطرح مخصصة تصنيف. رؤية الروبوت و بيثون / التوت بي عينة التطبيق للحصول على مثال نهاية إلى نهاية كيفية استخدام نموذج تصنيف TFLite قفة.

zip pose_classifier.zip pose_labels.txt pose_classifier.tflite
adding: pose_labels.txt (stored 0%)
  adding: pose_classifier.tflite (deflated 35%)
# Download the zip archive if running on Colab.
try:
  from google.colab import files
  files.download('pose_classifier.zip')
except:
  pass