Ayuda a proteger la Gran Barrera de Coral con TensorFlow en Kaggle Únete Challenge

Genera música con un RNN

Ver en TensorFlow.org Ejecutar en Google Colab Ver fuente en GitHub Descargar libreta

Este tutorial le muestra cómo generar notas musicales usando un RNN simple. Entrenará un modelo utilizando una colección de archivos MIDI de piano del conjunto de datos MAESTRO . Dada una secuencia de notas, su modelo aprenderá a predecir la próxima nota en la secuencia. Puede generar secuencias de notas más largas llamando al modelo repetidamente.

Este tutorial contiene código completo para analizar y crear archivos MIDI. Puede obtener más información sobre cómo funcionan los RNN visitando Generación de texto con un RNN .

Configuración

Este tutorial usa la biblioteca pretty_midi para crear y analizar archivos MIDI y pyfluidsynth para generar reproducción de audio en Colab.

sudo apt install -y fluidsynth
The following packages were automatically installed and are no longer required:
  linux-gcp-5.4-headers-5.4.0-1040 linux-gcp-5.4-headers-5.4.0-1043
  linux-gcp-5.4-headers-5.4.0-1044 linux-gcp-5.4-headers-5.4.0-1049
  linux-headers-5.4.0-1049-gcp linux-image-5.4.0-1049-gcp
  linux-modules-5.4.0-1049-gcp linux-modules-extra-5.4.0-1049-gcp
Use 'sudo apt autoremove' to remove them.
The following additional packages will be installed:
  fluid-soundfont-gm libasyncns0 libdouble-conversion1 libevdev2 libflac8
  libfluidsynth1 libgudev-1.0-0 libinput-bin libinput10 libjack-jackd2-0
  libmtdev1 libogg0 libpulse0 libqt5core5a libqt5dbus5 libqt5gui5
  libqt5network5 libqt5svg5 libqt5widgets5 libqt5x11extras5 libsamplerate0
  libsndfile1 libvorbis0a libvorbisenc2 libwacom-bin libwacom-common libwacom2
  libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0
  libxcb-render-util0 libxcb-shape0 libxcb-util1 libxcb-xinerama0 libxcb-xkb1
  libxkbcommon-x11-0 qsynth qt5-gtk-platformtheme qttranslations5-l10n
Suggested packages:
  fluid-soundfont-gs timidity jackd2 pulseaudio qt5-image-formats-plugins
  qtwayland5 jackd
The following NEW packages will be installed:
  fluid-soundfont-gm fluidsynth libasyncns0 libdouble-conversion1 libevdev2
  libflac8 libfluidsynth1 libgudev-1.0-0 libinput-bin libinput10
  libjack-jackd2-0 libmtdev1 libogg0 libpulse0 libqt5core5a libqt5dbus5
  libqt5gui5 libqt5network5 libqt5svg5 libqt5widgets5 libqt5x11extras5
  libsamplerate0 libsndfile1 libvorbis0a libvorbisenc2 libwacom-bin
  libwacom-common libwacom2 libxcb-icccm4 libxcb-image0 libxcb-keysyms1
  libxcb-randr0 libxcb-render-util0 libxcb-shape0 libxcb-util1
  libxcb-xinerama0 libxcb-xkb1 libxkbcommon-x11-0 qsynth qt5-gtk-platformtheme
  qttranslations5-l10n
0 upgraded, 41 newly installed, 0 to remove and 143 not upgraded.
Need to get 132 MB of archives.
After this operation, 198 MB of additional disk space will be used.
Get:1 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libogg0 amd64 1.3.2-1 [17.2 kB]
Get:2 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libdouble-conversion1 amd64 2.0.1-4ubuntu1 [33.0 kB]
Get:3 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5core5a amd64 5.9.5+dfsg-0ubuntu2.6 [2035 kB]
Get:4 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libevdev2 amd64 1.5.8+dfsg-1ubuntu0.1 [28.9 kB]
Get:5 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libmtdev1 amd64 1.1.5-1ubuntu3 [13.8 kB]
Get:6 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libgudev-1.0-0 amd64 1:232-2 [13.6 kB]
Get:7 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom-common all 0.29-1 [36.9 kB]
Get:8 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom2 amd64 0.29-1 [17.7 kB]
Get:9 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libinput-bin amd64 1.10.4-1ubuntu0.18.04.2 [11.2 kB]
Get:10 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libinput10 amd64 1.10.4-1ubuntu0.18.04.2 [86.2 kB]
Get:11 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5dbus5 amd64 5.9.5+dfsg-0ubuntu2.6 [195 kB]
Get:12 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5network5 amd64 5.9.5+dfsg-0ubuntu2.6 [634 kB]
Get:13 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-icccm4 amd64 0.4.1-1ubuntu1 [10.4 kB]
Get:14 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-util1 amd64 0.4.0-0ubuntu3 [11.2 kB]
Get:15 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-image0 amd64 0.4.0-1build1 [12.3 kB]
Get:16 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-keysyms1 amd64 0.4.0-1 [8406 B]
Get:17 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-randr0 amd64 1.13-2~ubuntu18.04 [16.4 kB]
Get:18 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libxcb-render-util0 amd64 0.3.9-1 [9638 B]
Get:19 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-shape0 amd64 1.13-2~ubuntu18.04 [5972 B]
Get:20 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-xinerama0 amd64 1.13-2~ubuntu18.04 [5264 B]
Get:21 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxcb-xkb1 amd64 1.13-2~ubuntu18.04 [30.1 kB]
Get:22 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxkbcommon-x11-0 amd64 0.8.2-1~ubuntu18.04.1 [13.4 kB]
Get:23 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5gui5 amd64 5.9.5+dfsg-0ubuntu2.6 [2568 kB]
Get:24 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqt5widgets5 amd64 5.9.5+dfsg-0ubuntu2.6 [2203 kB]
Get:25 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libqt5svg5 amd64 5.9.5-0ubuntu1 [128 kB]
Get:26 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 fluid-soundfont-gm all 3.1-5.1 [119 MB]
Get:27 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libsamplerate0 amd64 0.1.9-1 [938 kB]
Get:28 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libjack-jackd2-0 amd64 1.9.12~dfsg-2 [263 kB]
Get:29 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libasyncns0 amd64 0.8-6 [12.1 kB]
Get:30 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libflac8 amd64 1.3.2-1 [213 kB]
Get:31 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libvorbis0a amd64 1.3.5-4.2 [86.4 kB]
Get:32 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libvorbisenc2 amd64 1.3.5-4.2 [70.7 kB]
Get:33 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libsndfile1 amd64 1.0.28-4ubuntu0.18.04.2 [170 kB]
Get:34 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpulse0 amd64 1:11.1-1ubuntu7.11 [266 kB]
Get:35 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 libfluidsynth1 amd64 1.1.9-1 [137 kB]
Get:36 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 fluidsynth amd64 1.1.9-1 [20.7 kB]
Get:37 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 libqt5x11extras5 amd64 5.9.5-0ubuntu1 [8596 B]
Get:38 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 libwacom-bin amd64 0.29-1 [4712 B]
Get:39 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 qsynth amd64 0.5.0-2 [191 kB]
Get:40 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic-updates/main amd64 qt5-gtk-platformtheme amd64 5.9.5+dfsg-0ubuntu2.6 [117 kB]
Get:41 http://asia-east1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 qttranslations5-l10n all 5.9.5-0ubuntu1 [1485 kB]
Fetched 132 MB in 7s (19.1 MB/s)
Extracting templates from packages: 100%

7[0;23r8[1ASelecting previously unselected package libogg0:amd64.
(Reading database ... 281948 files and directories currently installed.)
Preparing to unpack .../00-libogg0_1.3.2-1_amd64.deb ...
7[24;0fProgress: [  0%] [..........................................................] 8Unpacking libogg0:amd64 (1.3.2-1) ...
7[24;0fProgress: [  1%] [..........................................................] 8Selecting previously unselected package libdouble-conversion1:amd64.
Preparing to unpack .../01-libdouble-conversion1_2.0.1-4ubuntu1_amd64.deb ...
Unpacking libdouble-conversion1:amd64 (2.0.1-4ubuntu1) ...
7[24;0fProgress: [  2%] [#.........................................................] 8Selecting previously unselected package libqt5core5a:amd64.
Preparing to unpack .../02-libqt5core5a_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
7[24;0fProgress: [  3%] [#.........................................................] 8Unpacking libqt5core5a:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [  4%] [##........................................................] 8Selecting previously unselected package libevdev2:amd64.
Preparing to unpack .../03-libevdev2_1.5.8+dfsg-1ubuntu0.1_amd64.deb ...
Unpacking libevdev2:amd64 (1.5.8+dfsg-1ubuntu0.1) ...
7[24;0fProgress: [  5%] [###.......................................................] 8Selecting previously unselected package libmtdev1:amd64.
Preparing to unpack .../04-libmtdev1_1.1.5-1ubuntu3_amd64.deb ...
7[24;0fProgress: [  6%] [###.......................................................] 8Unpacking libmtdev1:amd64 (1.1.5-1ubuntu3) ...
7[24;0fProgress: [  7%] [####......................................................] 8Selecting previously unselected package libgudev-1.0-0:amd64.
Preparing to unpack .../05-libgudev-1.0-0_1%3a232-2_amd64.deb ...
Unpacking libgudev-1.0-0:amd64 (1:232-2) ...
7[24;0fProgress: [  8%] [####......................................................] 8Selecting previously unselected package libwacom-common.
Preparing to unpack .../06-libwacom-common_0.29-1_all.deb ...
7[24;0fProgress: [  9%] [#####.....................................................] 8Unpacking libwacom-common (0.29-1) ...
7[24;0fProgress: [ 10%] [#####.....................................................] 8Selecting previously unselected package libwacom2:amd64.
Preparing to unpack .../07-libwacom2_0.29-1_amd64.deb ...
Unpacking libwacom2:amd64 (0.29-1) ...
7[24;0fProgress: [ 11%] [######....................................................] 8Selecting previously unselected package libinput-bin.
Preparing to unpack .../08-libinput-bin_1.10.4-1ubuntu0.18.04.2_amd64.deb ...
7[24;0fProgress: [ 12%] [#######...................................................] 8Unpacking libinput-bin (1.10.4-1ubuntu0.18.04.2) ...
7[24;0fProgress: [ 13%] [#######...................................................] 8Selecting previously unselected package libinput10:amd64.
Preparing to unpack .../09-libinput10_1.10.4-1ubuntu0.18.04.2_amd64.deb ...
Unpacking libinput10:amd64 (1.10.4-1ubuntu0.18.04.2) ...
7[24;0fProgress: [ 14%] [########..................................................] 8Selecting previously unselected package libqt5dbus5:amd64.
Preparing to unpack .../10-libqt5dbus5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
7[24;0fProgress: [ 15%] [########..................................................] 8Unpacking libqt5dbus5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 16%] [#########.................................................] 8Selecting previously unselected package libqt5network5:amd64.
Preparing to unpack .../11-libqt5network5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
Unpacking libqt5network5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 17%] [##########................................................] 8Selecting previously unselected package libxcb-icccm4:amd64.
Preparing to unpack .../12-libxcb-icccm4_0.4.1-1ubuntu1_amd64.deb ...
Unpacking libxcb-icccm4:amd64 (0.4.1-1ubuntu1) ...
7[24;0fProgress: [ 18%] [##########................................................] 8Selecting previously unselected package libxcb-util1:amd64.
Preparing to unpack .../13-libxcb-util1_0.4.0-0ubuntu3_amd64.deb ...
7[24;0fProgress: [ 19%] [###########...............................................] 8Unpacking libxcb-util1:amd64 (0.4.0-0ubuntu3) ...
7[24;0fProgress: [ 20%] [###########...............................................] 8Selecting previously unselected package libxcb-image0:amd64.
Preparing to unpack .../14-libxcb-image0_0.4.0-1build1_amd64.deb ...
Unpacking libxcb-image0:amd64 (0.4.0-1build1) ...
7[24;0fProgress: [ 21%] [############..............................................] 8Selecting previously unselected package libxcb-keysyms1:amd64.
Preparing to unpack .../15-libxcb-keysyms1_0.4.0-1_amd64.deb ...
7[24;0fProgress: [ 22%] [############..............................................] 8Unpacking libxcb-keysyms1:amd64 (0.4.0-1) ...
7[24;0fProgress: [ 23%] [#############.............................................] 8Selecting previously unselected package libxcb-randr0:amd64.
Preparing to unpack .../16-libxcb-randr0_1.13-2~ubuntu18.04_amd64.deb ...
Unpacking libxcb-randr0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 24%] [##############............................................] 8Selecting previously unselected package libxcb-render-util0:amd64.
Preparing to unpack .../17-libxcb-render-util0_0.3.9-1_amd64.deb ...
7[24;0fProgress: [ 25%] [##############............................................] 8Unpacking libxcb-render-util0:amd64 (0.3.9-1) ...
7[24;0fProgress: [ 26%] [###############...........................................] 8Selecting previously unselected package libxcb-shape0:amd64.
Preparing to unpack .../18-libxcb-shape0_1.13-2~ubuntu18.04_amd64.deb ...
Unpacking libxcb-shape0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 27%] [###############...........................................] 8Selecting previously unselected package libxcb-xinerama0:amd64.
Preparing to unpack .../19-libxcb-xinerama0_1.13-2~ubuntu18.04_amd64.deb ...
7[24;0fProgress: [ 28%] [################..........................................] 8Unpacking libxcb-xinerama0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 29%] [################..........................................] 8Selecting previously unselected package libxcb-xkb1:amd64.
Preparing to unpack .../20-libxcb-xkb1_1.13-2~ubuntu18.04_amd64.deb ...
Unpacking libxcb-xkb1:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 30%] [#################.........................................] 8Selecting previously unselected package libxkbcommon-x11-0:amd64.
Preparing to unpack .../21-libxkbcommon-x11-0_0.8.2-1~ubuntu18.04.1_amd64.deb ...
7[24;0fProgress: [ 31%] [##################........................................] 8Unpacking libxkbcommon-x11-0:amd64 (0.8.2-1~ubuntu18.04.1) ...
7[24;0fProgress: [ 32%] [##################........................................] 8Selecting previously unselected package libqt5gui5:amd64.
Preparing to unpack .../22-libqt5gui5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
Unpacking libqt5gui5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 33%] [###################.......................................] 8Selecting previously unselected package libqt5widgets5:amd64.
Preparing to unpack .../23-libqt5widgets5_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
Unpacking libqt5widgets5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 34%] [###################.......................................] 8Selecting previously unselected package libqt5svg5:amd64.
Preparing to unpack .../24-libqt5svg5_5.9.5-0ubuntu1_amd64.deb ...
7[24;0fProgress: [ 35%] [####################......................................] 8Unpacking libqt5svg5:amd64 (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 36%] [#####################.....................................] 8Selecting previously unselected package fluid-soundfont-gm.
Preparing to unpack .../25-fluid-soundfont-gm_3.1-5.1_all.deb ...
Unpacking fluid-soundfont-gm (3.1-5.1) ...
7[24;0fProgress: [ 37%] [#####################.....................................] 8Selecting previously unselected package libsamplerate0:amd64.
Preparing to unpack .../26-libsamplerate0_0.1.9-1_amd64.deb ...
7[24;0fProgress: [ 38%] [######################....................................] 8Unpacking libsamplerate0:amd64 (0.1.9-1) ...
7[24;0fProgress: [ 39%] [######################....................................] 8Selecting previously unselected package libjack-jackd2-0:amd64.
Preparing to unpack .../27-libjack-jackd2-0_1.9.12~dfsg-2_amd64.deb ...
Unpacking libjack-jackd2-0:amd64 (1.9.12~dfsg-2) ...
7[24;0fProgress: [ 40%] [#######################...................................] 8Selecting previously unselected package libasyncns0:amd64.
Preparing to unpack .../28-libasyncns0_0.8-6_amd64.deb ...
7[24;0fProgress: [ 41%] [#######################...................................] 8Unpacking libasyncns0:amd64 (0.8-6) ...
7[24;0fProgress: [ 42%] [########################..................................] 8Selecting previously unselected package libflac8:amd64.
Preparing to unpack .../29-libflac8_1.3.2-1_amd64.deb ...
Unpacking libflac8:amd64 (1.3.2-1) ...
7[24;0fProgress: [ 43%] [#########################.................................] 8Selecting previously unselected package libvorbis0a:amd64.
Preparing to unpack .../30-libvorbis0a_1.3.5-4.2_amd64.deb ...
7[24;0fProgress: [ 44%] [#########################.................................] 8Unpacking libvorbis0a:amd64 (1.3.5-4.2) ...
7[24;0fProgress: [ 45%] [##########################................................] 8Selecting previously unselected package libvorbisenc2:amd64.
Preparing to unpack .../31-libvorbisenc2_1.3.5-4.2_amd64.deb ...
Unpacking libvorbisenc2:amd64 (1.3.5-4.2) ...
7[24;0fProgress: [ 46%] [##########################................................] 8Selecting previously unselected package libsndfile1:amd64.
Preparing to unpack .../32-libsndfile1_1.0.28-4ubuntu0.18.04.2_amd64.deb ...
7[24;0fProgress: [ 47%] [###########################...............................] 8Unpacking libsndfile1:amd64 (1.0.28-4ubuntu0.18.04.2) ...
7[24;0fProgress: [ 48%] [###########################...............................] 8Selecting previously unselected package libpulse0:amd64.
Preparing to unpack .../33-libpulse0_1%3a11.1-1ubuntu7.11_amd64.deb ...
Unpacking libpulse0:amd64 (1:11.1-1ubuntu7.11) ...
7[24;0fProgress: [ 49%] [############################..............................] 8Selecting previously unselected package libfluidsynth1:amd64.
Preparing to unpack .../34-libfluidsynth1_1.1.9-1_amd64.deb ...
7[24;0fProgress: [ 50%] [#############################.............................] 8Unpacking libfluidsynth1:amd64 (1.1.9-1) ...
Selecting previously unselected package fluidsynth.
Preparing to unpack .../35-fluidsynth_1.1.9-1_amd64.deb ...
7[24;0fProgress: [ 51%] [#############################.............................] 8Unpacking fluidsynth (1.1.9-1) ...
7[24;0fProgress: [ 52%] [##############################............................] 8Selecting previously unselected package libqt5x11extras5:amd64.
Preparing to unpack .../36-libqt5x11extras5_5.9.5-0ubuntu1_amd64.deb ...
Unpacking libqt5x11extras5:amd64 (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 53%] [##############################............................] 8Selecting previously unselected package libwacom-bin.
Preparing to unpack .../37-libwacom-bin_0.29-1_amd64.deb ...
7[24;0fProgress: [ 54%] [###############################...........................] 8Unpacking libwacom-bin (0.29-1) ...
7[24;0fProgress: [ 55%] [################################..........................] 8Selecting previously unselected package qsynth.
Preparing to unpack .../38-qsynth_0.5.0-2_amd64.deb ...
Unpacking qsynth (0.5.0-2) ...
7[24;0fProgress: [ 56%] [################################..........................] 8Selecting previously unselected package qt5-gtk-platformtheme:amd64.
Preparing to unpack .../39-qt5-gtk-platformtheme_5.9.5+dfsg-0ubuntu2.6_amd64.deb ...
7[24;0fProgress: [ 57%] [#################################.........................] 8Unpacking qt5-gtk-platformtheme:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 58%] [#################################.........................] 8Selecting previously unselected package qttranslations5-l10n.
Preparing to unpack .../40-qttranslations5-l10n_5.9.5-0ubuntu1_all.deb ...
Unpacking qttranslations5-l10n (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 59%] [##################################........................] 8Setting up libxcb-xinerama0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 60%] [##################################........................] 8Setting up libxcb-render-util0:amd64 (0.3.9-1) ...
7[24;0fProgress: [ 61%] [###################################.......................] 8Setting up libxcb-randr0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 62%] [####################################......................] 8Setting up libxcb-icccm4:amd64 (0.4.1-1ubuntu1) ...
7[24;0fProgress: [ 63%] [####################################......................] 8Setting up libasyncns0:amd64 (0.8-6) ...
7[24;0fProgress: [ 64%] [#####################################.....................] 8Setting up libwacom-common (0.29-1) ...
7[24;0fProgress: [ 65%] [#####################################.....................] 8Setting up libdouble-conversion1:amd64 (2.0.1-4ubuntu1) ...
7[24;0fProgress: [ 66%] [######################################....................] 8Setting up libevdev2:amd64 (1.5.8+dfsg-1ubuntu0.1) ...
7[24;0fProgress: [ 67%] [#######################################...................] 8Setting up fluid-soundfont-gm (3.1-5.1) ...
7[24;0fProgress: [ 68%] [#######################################...................] 8Setting up libxcb-util1:amd64 (0.4.0-0ubuntu3) ...
7[24;0fProgress: [ 69%] [########################################..................] 8Setting up libogg0:amd64 (1.3.2-1) ...
7[24;0fProgress: [ 70%] [########################################..................] 8Setting up qttranslations5-l10n (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 71%] [#########################################.................] 8Setting up libmtdev1:amd64 (1.1.5-1ubuntu3) ...
7[24;0fProgress: [ 72%] [#########################################.................] 8Setting up libxcb-shape0:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 73%] [##########################################................] 8Setting up libgudev-1.0-0:amd64 (1:232-2) ...
7[24;0fProgress: [ 74%] [###########################################...............] 8Setting up libxcb-keysyms1:amd64 (0.4.0-1) ...
7[24;0fProgress: [ 75%] [###########################################...............] 8Setting up libsamplerate0:amd64 (0.1.9-1) ...
7[24;0fProgress: [ 76%] [############################################..............] 8Setting up libvorbis0a:amd64 (1.3.5-4.2) ...
7[24;0fProgress: [ 77%] [############################################..............] 8Setting up libxcb-xkb1:amd64 (1.13-2~ubuntu18.04) ...
7[24;0fProgress: [ 78%] [#############################################.............] 8Setting up libqt5core5a:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 79%] [#############################################.............] 8Setting up libqt5dbus5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 80%] [##############################################............] 8Setting up libqt5network5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 81%] [###############################################...........] 8Setting up libwacom2:amd64 (0.29-1) ...
7[24;0fProgress: [ 82%] [###############################################...........] 8Setting up libxcb-image0:amd64 (0.4.0-1build1) ...
7[24;0fProgress: [ 83%] [################################################..........] 8Setting up libflac8:amd64 (1.3.2-1) ...
Setting up libinput-bin (1.10.4-1ubuntu0.18.04.2) ...
7[24;0fProgress: [ 84%] [################################################..........] 8Setting up libxkbcommon-x11-0:amd64 (0.8.2-1~ubuntu18.04.1) ...
7[24;0fProgress: [ 85%] [#################################################.........] 8Setting up libwacom-bin (0.29-1) ...
7[24;0fProgress: [ 86%] [##################################################........] 8Setting up libjack-jackd2-0:amd64 (1.9.12~dfsg-2) ...
7[24;0fProgress: [ 87%] [##################################################........] 8Setting up libvorbisenc2:amd64 (1.3.5-4.2) ...
7[24;0fProgress: [ 88%] [###################################################.......] 8Setting up libinput10:amd64 (1.10.4-1ubuntu0.18.04.2) ...
7[24;0fProgress: [ 89%] [###################################################.......] 8Setting up libsndfile1:amd64 (1.0.28-4ubuntu0.18.04.2) ...
7[24;0fProgress: [ 90%] [####################################################......] 8Setting up libqt5gui5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 91%] [####################################################......] 8Setting up qt5-gtk-platformtheme:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 92%] [#####################################################.....] 8Setting up libqt5x11extras5:amd64 (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 93%] [######################################################....] 8Setting up libqt5widgets5:amd64 (5.9.5+dfsg-0ubuntu2.6) ...
7[24;0fProgress: [ 94%] [######################################################....] 8Setting up libpulse0:amd64 (1:11.1-1ubuntu7.11) ...
7[24;0fProgress: [ 95%] [#######################################################...] 8Setting up libqt5svg5:amd64 (5.9.5-0ubuntu1) ...
7[24;0fProgress: [ 96%] [#######################################################...] 8Setting up libfluidsynth1:amd64 (1.1.9-1) ...
7[24;0fProgress: [ 97%] [########################################################..] 8Setting up fluidsynth (1.1.9-1) ...
7[24;0fProgress: [ 98%] [########################################################..] 8Setting up qsynth (0.5.0-2) ...
7[24;0fProgress: [ 99%] [#########################################################.] 8Processing triggers for hicolor-icon-theme (0.17-2) ...
Processing triggers for mime-support (3.60ubuntu1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.2) ...
Processing triggers for udev (237-3ubuntu10.50) ...
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...

7[0;24r8[1A[J
pip install --upgrade pyfluidsynth
pip install pretty_midi
import collections
import datetime
import fluidsynth
import glob
import numpy as np
import pathlib
import pandas as pd
import pretty_midi
import seaborn as sns
import tensorflow as tf

from IPython import display
from matplotlib import pyplot as plt
from typing import Dict, List, Optional, Sequence, Tuple
seed = 42
tf.random.set_seed(seed)
np.random.seed(seed)

# Sampling rate for audio playback
_SAMPLING_RATE = 16000

Descargar el conjunto de datos de Maestro

data_dir = pathlib.Path('data/maestro-v2.0.0')
if not data_dir.exists():
  tf.keras.utils.get_file(
      'maestro-v2.0.0-midi.zip',
      origin='https://storage.googleapis.com/magentadata/datasets/maestro/v2.0.0/maestro-v2.0.0-midi.zip',
      extract=True,
      cache_dir='.', cache_subdir='data',
  )
Downloading data from https://storage.googleapis.com/magentadata/datasets/maestro/v2.0.0/maestro-v2.0.0-midi.zip
59244544/59243107 [==============================] - 1s 0us/step
59252736/59243107 [==============================] - 1s 0us/step

El conjunto de datos contiene alrededor de 1200 archivos MIDI.

filenames = glob.glob(str(data_dir/'**/*.mid*'))
print('Number of files:', len(filenames))
Number of files: 1282

Procesar un archivo MIDI

Primero, usa pretty_midi para analizar un solo archivo MIDI e inspeccionar el formato de las notas. Si desea descargar el archivo MIDI a continuación para reproducirlo en su computadora, puede hacerlo en colab escribiendo files.download(sample_file) .

sample_file = filenames[1]
print(sample_file)
data/maestro-v2.0.0/2006/MIDI-Unprocessed_01_R1_2006_01-09_ORIG_MID--AUDIO_01_R1_2006_02_Track02_wav.midi

Genere un objeto PrettyMIDI para el archivo MIDI de muestra.

pm = pretty_midi.PrettyMIDI(sample_file)

Reproduzca el archivo de muestra. El widget de reproducción puede tardar varios segundos en cargarse.

def display_audio(pm: pretty_midi.PrettyMIDI, seconds=30):
  waveform = pm.fluidsynth(fs=_SAMPLING_RATE)
  # Take a sample of the generated waveform to mitigate kernel resets
  waveform_short = waveform[:seconds*_SAMPLING_RATE]
  return display.Audio(waveform_short, rate=_SAMPLING_RATE)
display_audio(pm)

Inspeccione un poco el archivo MIDI. ¿Qué tipo de instrumentos se utilizan?

print('Number of instruments:', len(pm.instruments))
instrument = pm.instruments[0]
instrument_name = pretty_midi.program_to_instrument_name(instrument.program)
print('Instrument name:', instrument_name)
Number of instruments: 1
Instrument name: Acoustic Grand Piano

Extraer notas

for i, note in enumerate(instrument.notes[:10]):
  note_name = pretty_midi.note_number_to_name(note.pitch)
  duration = note.end - note.start
  print(f'{i}: pitch={note.pitch}, note_name={note_name},'
        f' duration={duration:.4f}')
0: pitch=79, note_name=G5, duration=0.3047
1: pitch=74, note_name=D5, duration=0.0977
2: pitch=71, note_name=B4, duration=0.0807
3: pitch=67, note_name=G4, duration=0.0482
4: pitch=74, note_name=D5, duration=0.0508
5: pitch=72, note_name=C5, duration=0.2227
6: pitch=69, note_name=A4, duration=0.0521
7: pitch=62, note_name=D4, duration=0.0547
8: pitch=72, note_name=C5, duration=0.0521
9: pitch=71, note_name=B4, duration=0.2201

Utilizará tres variables para representar una nota cuando entrene el modelo: pitch , step y duration . El tono es la calidad perceptible del sonido como un número de nota MIDI. El step es el tiempo transcurrido desde la nota anterior o el inicio de la pista. La duration es cuánto tiempo se reproducirá la nota en segundos y es la diferencia entre el final de la nota y los tiempos de inicio de la nota.

Extraiga las notas del archivo MIDI de muestra.

def midi_to_notes(midi_file: str) -> pd.DataFrame:
  pm = pretty_midi.PrettyMIDI(midi_file)
  instrument = pm.instruments[0]
  notes = collections.defaultdict(list)

  # Sort the notes by start time
  sorted_notes = sorted(instrument.notes, key=lambda note: note.start)
  prev_start = sorted_notes[0].start

  for note in sorted_notes:
    start = note.start
    end = note.end
    notes['pitch'].append(note.pitch)
    notes['start'].append(start)
    notes['end'].append(end)
    notes['step'].append(start - prev_start)
    notes['duration'].append(end - start)
    prev_start = start

  return pd.DataFrame({name: np.array(value) for name, value in notes.items()})
raw_notes = midi_to_notes(sample_file)
raw_notes.head()

Puede ser más fácil interpretar los nombres de las notas en lugar de los tonos, por lo que puede usar la función a continuación para convertir los valores de tono numéricos en nombres de notas. El nombre de la nota muestra el tipo de nota, la alteración y el número de octava (por ejemplo, C#4).

get_note_names = np.vectorize(pretty_midi.note_number_to_name)
sample_note_names = get_note_names(raw_notes['pitch'])
sample_note_names[:10]
array(['G5', 'D5', 'B4', 'G4', 'D5', 'C5', 'A4', 'D4', 'C5', 'B4'],
      dtype='<U3')

Para visualizar la pieza musical, marque el tono de la nota, el comienzo y el final a lo largo de la pista (es decir, piano roll). Comience con las primeras 100 notas

def plot_piano_roll(notes: pd.DataFrame, count: Optional[int] = None):
  if count:
    title = f'First {count} notes'
  else:
    title = f'Whole track'
    count = len(notes['pitch'])
  plt.figure(figsize=(20, 4))
  plot_pitch = np.stack([notes['pitch'], notes['pitch']], axis=0)
  plot_start_stop = np.stack([notes['start'], notes['end']], axis=0)
  plt.plot(
      plot_start_stop[:, :count], plot_pitch[:, :count], color="b", marker=".")
  plt.xlabel('Time [s]')
  plt.ylabel('Pitch')
  _ = plt.title(title)
plot_piano_roll(raw_notes, count=100)

png

Trace las notas de toda la pista.

plot_piano_roll(raw_notes)

png

Consulta la distribución de cada nota variable.

def plot_distributions(notes: pd.DataFrame, drop_percentile=2.5):
  plt.figure(figsize=[15, 5])
  plt.subplot(1, 3, 1)
  sns.histplot(notes, x="pitch", bins=20)

  plt.subplot(1, 3, 2)
  max_step = np.percentile(notes['step'], 100 - drop_percentile)
  sns.histplot(notes, x="step", bins=np.linspace(0, max_step, 21))

  plt.subplot(1, 3, 3)
  max_duration = np.percentile(notes['duration'], 100 - drop_percentile)
  sns.histplot(notes, x="duration", bins=np.linspace(0, max_duration, 21))
plot_distributions(raw_notes)

png

Crear un archivo MIDI

Puede generar su propio archivo MIDI a partir de una lista de notas usando la función a continuación.

def notes_to_midi(
  notes: pd.DataFrame,
  out_file: str, 
  instrument_name: str,
  velocity: int = 100,  # note loudness
) -> pretty_midi.PrettyMIDI:

  pm = pretty_midi.PrettyMIDI()
  instrument = pretty_midi.Instrument(
      program=pretty_midi.instrument_name_to_program(
          instrument_name))

  prev_start = 0
  for i, note in notes.iterrows():
    start = float(prev_start + note['step'])
    end = float(start + note['duration'])
    note = pretty_midi.Note(
        velocity=velocity,
        pitch=int(note['pitch']),
        start=start,
        end=end,
    )
    instrument.notes.append(note)
    prev_start = start

  pm.instruments.append(instrument)
  pm.write(out_file)
  return pm
example_file = 'example.midi'
example_pm = notes_to_midi(
    raw_notes, out_file=example_file, instrument_name=instrument_name)

Reproduzca el archivo MIDI generado y vea si hay alguna diferencia.

display_audio(example_pm)

Como antes, puede escribir files.download(example_file) para descargar y reproducir este archivo.

Crear el conjunto de datos de entrenamiento

Cree el conjunto de datos de entrenamiento extrayendo notas de los archivos MIDI. Puede comenzar usando una pequeña cantidad de archivos y experimentar más tarde con más. Esto puede tomar un par de minutos.

num_files = 5
all_notes = []
for f in filenames[:num_files]:
  notes = midi_to_notes(f)
  all_notes.append(notes)

all_notes = pd.concat(all_notes)
n_notes = len(all_notes)
print('Number of notes parsed:', n_notes)
Number of notes parsed: 32447

A continuación, cree un tf.data.Dataset a partir de las notas analizadas.

key_order = ['pitch', 'step', 'duration']
train_notes = np.stack([all_notes[key] for key in key_order], axis=1)
notes_ds = tf.data.Dataset.from_tensor_slices(train_notes)
notes_ds.element_spec
TensorSpec(shape=(3,), dtype=tf.float64, name=None)

Entrenará el modelo en lotes de secuencias de notas. Cada ejemplo constará de una secuencia de notas como características de entrada y la siguiente nota como etiqueta. De esta forma, el modelo será entrenado para predecir la siguiente nota en una secuencia. Puede encontrar un diagrama que explica este proceso (y más detalles) en Clasificación de texto con un RNN .

Puede usar la práctica función de ventana con tamaño seq_length para crear las características y etiquetas en este formato.

def create_sequences(
    dataset: tf.data.Dataset, 
    seq_length: int,
    vocab_size = 128,
) -> tf.data.Dataset:
  """Returns TF Dataset of sequence and label examples."""
  seq_length = seq_length+1

  # Take 1 extra for the labels
  windows = dataset.window(seq_length, shift=1, stride=1,
                              drop_remainder=True)

  # `flat_map` flattens the" dataset of datasets" into a dataset of tensors
  flatten = lambda x: x.batch(seq_length, drop_remainder=True)
  sequences = windows.flat_map(flatten)

  # Normalize note pitch
  def scale_pitch(x):
    x = x/[vocab_size,1.0,1.0]
    return x

  # Split the labels
  def split_labels(sequences):
    inputs = sequences[:-1]
    labels_dense = sequences[-1]
    labels = {key:labels_dense[i] for i,key in enumerate(key_order)}

    return scale_pitch(inputs), labels

  return sequences.map(split_labels, num_parallel_calls=tf.data.AUTOTUNE)

Establezca la longitud de la secuencia para cada ejemplo. Experimente con diferentes longitudes (p. ej., 50, 100, 150) para ver cuál funciona mejor para los datos, o utilice el ajuste de hiperparámetros . El tamaño del vocabulario ( vocab_size ) se establece en 128 y representa todos los tonos admitidos por pretty_midi .

seq_length = 25
vocab_size = 128
seq_ds = create_sequences(notes_ds, seq_length, vocab_size)
seq_ds.element_spec
(TensorSpec(shape=(25, 3), dtype=tf.float64, name=None),
 {'pitch': TensorSpec(shape=(), dtype=tf.float64, name=None),
  'step': TensorSpec(shape=(), dtype=tf.float64, name=None),
  'duration': TensorSpec(shape=(), dtype=tf.float64, name=None)})

La forma del conjunto de datos es (100,1) , lo que significa que el modelo tomará 100 notas como entrada y aprenderá a predecir la siguiente nota como salida.

for seq, target in seq_ds.take(1):
  print('sequence shape:', seq.shape)
  print('sequence elements (first 10):', seq[0: 10])
  print()
  print('target:', target)
sequence shape: (25, 3)
sequence elements (first 10): tf.Tensor(
[[0.46875    0.         0.36197917]
 [0.375      0.01822917 0.05989583]
 [0.28125    0.00130208 0.04817708]
 [0.4921875  0.66276042 0.05729167]
 [0.3046875  0.00911458 0.0546875 ]
 [0.3984375  0.         0.06770833]
 [0.3359375  0.35546875 0.0390625 ]
 [0.4296875  0.00260417 0.0546875 ]
 [0.5234375  0.00390625 0.08723958]
 [0.375      0.35026042 0.03645833]], shape=(10, 3), dtype=float64)

target: {'pitch': <tf.Tensor: shape=(), dtype=float64, numpy=72.0>, 'step': <tf.Tensor: shape=(), dtype=float64, numpy=0.35546875>, 'duration': <tf.Tensor: shape=(), dtype=float64, numpy=0.1171875>}

Agrupe los ejemplos y configure el conjunto de datos para el rendimiento.

batch_size = 64
buffer_size = n_notes - seq_length  # the number of items in the dataset
train_ds = (seq_ds
            .shuffle(buffer_size)
            .batch(batch_size, drop_remainder=True)
            .cache()
            .prefetch(tf.data.experimental.AUTOTUNE))
train_ds.element_spec
(TensorSpec(shape=(64, 25, 3), dtype=tf.float64, name=None),
 {'pitch': TensorSpec(shape=(64,), dtype=tf.float64, name=None),
  'step': TensorSpec(shape=(64,), dtype=tf.float64, name=None),
  'duration': TensorSpec(shape=(64,), dtype=tf.float64, name=None)})

Crear y entrenar el modelo.

El modelo tendrá tres salidas, una para cada variable de nota. Para pitch y duration , utilizará una función de pérdida personalizada basada en el error cuadrático medio que anima al modelo a generar valores no negativos.

def mse_with_positive_pressure(y_true: tf.Tensor, y_pred: tf.Tensor):
  mse = (y_true - y_pred) ** 2
  positive_pressure = 10 * tf.maximum(-y_pred, 0.0)
  return tf.reduce_mean(mse + positive_pressure)
input_shape = (seq_length, 3)
learning_rate = 0.005

inputs = tf.keras.Input(input_shape)
x = tf.keras.layers.LSTM(128)(inputs)

outputs = {
  'pitch': tf.keras.layers.Dense(128, name='pitch')(x),
  'step': tf.keras.layers.Dense(1, name='step')(x),
  'duration': tf.keras.layers.Dense(1, name='duration')(x),
}

model = tf.keras.Model(inputs, outputs)

loss = {
      'pitch': tf.keras.losses.SparseCategoricalCrossentropy(
          from_logits=True),
      'step': mse_with_positive_pressure,
      'duration': mse_with_positive_pressure,
}

optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)

model.compile(loss=loss, optimizer=optimizer)

model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 25, 3)]      0                                            
__________________________________________________________________________________________________
lstm (LSTM)                     (None, 128)          67584       input_1[0][0]                    
__________________________________________________________________________________________________
duration (Dense)                (None, 1)            129         lstm[0][0]                       
__________________________________________________________________________________________________
pitch (Dense)                   (None, 128)          16512       lstm[0][0]                       
__________________________________________________________________________________________________
step (Dense)                    (None, 1)            129         lstm[0][0]                       
==================================================================================================
Total params: 84,354
Trainable params: 84,354
Non-trainable params: 0
__________________________________________________________________________________________________

Al probar la función model.evaluate , puede ver que la pérdida de pitch es significativamente mayor que las pérdidas de step y duration . Tenga en cuenta que la loss es la pérdida total calculada sumando todas las demás pérdidas y actualmente está dominada por la pérdida de pitch .

losses = model.evaluate(train_ds, return_dict=True)
losses
506/506 [==============================] - 9s 3ms/step - loss: 5.1169 - duration_loss: 0.1823 - pitch_loss: 4.8491 - step_loss: 0.0855
{'loss': 5.116912364959717,
 'duration_loss': 0.18227165937423706,
 'pitch_loss': 4.849145889282227,
 'step_loss': 0.08549385517835617}

Una forma de equilibrar esto es usar el argumento loss_weights para compilar:

model.compile(
    loss=loss,
    loss_weights={
        'pitch': 0.05,
        'step': 1.0,
        'duration':1.0,
    },
    optimizer=optimizer,
)

La loss se convierte entonces en la suma ponderada de las pérdidas individuales.

model.evaluate(train_ds, return_dict=True)
506/506 [==============================] - 2s 3ms/step - loss: 0.5102 - duration_loss: 0.1823 - pitch_loss: 4.8491 - step_loss: 0.0855
{'loss': 0.5102230310440063,
 'duration_loss': 0.18227165937423706,
 'pitch_loss': 4.849145889282227,
 'step_loss': 0.08549385517835617}

Entrena al modelo.

callbacks = [
    tf.keras.callbacks.ModelCheckpoint(
        filepath='./training_checkpoints/ckpt_{epoch}',
        save_weights_only=True),
    tf.keras.callbacks.EarlyStopping(
        monitor='loss',
        patience=5,
        verbose=1,
        restore_best_weights=True),
]
%%time
epochs = 50

history = model.fit(
    train_ds,
    epochs=epochs,
    callbacks=callbacks,
)
Epoch 1/50
506/506 [==============================] - 4s 5ms/step - loss: 0.3957 - duration_loss: 0.1146 - pitch_loss: 4.0181 - step_loss: 0.0802
Epoch 2/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3851 - duration_loss: 0.1106 - pitch_loss: 3.9096 - step_loss: 0.0790
Epoch 3/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3840 - duration_loss: 0.1105 - pitch_loss: 3.8942 - step_loss: 0.0788
Epoch 4/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3809 - duration_loss: 0.1101 - pitch_loss: 3.8492 - step_loss: 0.0783
Epoch 5/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3770 - duration_loss: 0.1092 - pitch_loss: 3.7909 - step_loss: 0.0782
Epoch 6/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3747 - duration_loss: 0.1083 - pitch_loss: 3.7758 - step_loss: 0.0776
Epoch 7/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3732 - duration_loss: 0.1077 - pitch_loss: 3.7737 - step_loss: 0.0768
Epoch 8/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3717 - duration_loss: 0.1070 - pitch_loss: 3.7626 - step_loss: 0.0766
Epoch 9/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3695 - duration_loss: 0.1069 - pitch_loss: 3.7708 - step_loss: 0.0740
Epoch 10/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3712 - duration_loss: 0.1067 - pitch_loss: 3.7614 - step_loss: 0.0764
Epoch 11/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3666 - duration_loss: 0.1050 - pitch_loss: 3.7740 - step_loss: 0.0730
Epoch 12/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3688 - duration_loss: 0.1053 - pitch_loss: 3.7671 - step_loss: 0.0752
Epoch 13/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3633 - duration_loss: 0.1047 - pitch_loss: 3.7829 - step_loss: 0.0694
Epoch 14/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3657 - duration_loss: 0.1055 - pitch_loss: 3.8003 - step_loss: 0.0702
Epoch 15/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3679 - duration_loss: 0.1041 - pitch_loss: 3.7637 - step_loss: 0.0757
Epoch 16/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3555 - duration_loss: 0.1026 - pitch_loss: 3.7509 - step_loss: 0.0653
Epoch 17/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3553 - duration_loss: 0.1021 - pitch_loss: 3.7718 - step_loss: 0.0646
Epoch 18/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3495 - duration_loss: 0.1007 - pitch_loss: 3.7835 - step_loss: 0.0597
Epoch 19/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3471 - duration_loss: 0.1009 - pitch_loss: 3.7438 - step_loss: 0.0590
Epoch 20/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3404 - duration_loss: 0.0987 - pitch_loss: 3.7468 - step_loss: 0.0544
Epoch 21/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3548 - duration_loss: 0.0996 - pitch_loss: 3.7577 - step_loss: 0.0673
Epoch 22/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3767 - duration_loss: 0.1019 - pitch_loss: 3.8789 - step_loss: 0.0809
Epoch 23/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3354 - duration_loss: 0.0976 - pitch_loss: 3.7340 - step_loss: 0.0511
Epoch 24/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3346 - duration_loss: 0.0961 - pitch_loss: 3.7231 - step_loss: 0.0524
Epoch 25/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3297 - duration_loss: 0.0945 - pitch_loss: 3.7139 - step_loss: 0.0495
Epoch 26/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3285 - duration_loss: 0.0936 - pitch_loss: 3.7345 - step_loss: 0.0481
Epoch 27/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3296 - duration_loss: 0.0952 - pitch_loss: 3.7418 - step_loss: 0.0473
Epoch 28/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3253 - duration_loss: 0.0942 - pitch_loss: 3.7205 - step_loss: 0.0450
Epoch 29/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3234 - duration_loss: 0.0904 - pitch_loss: 3.7089 - step_loss: 0.0475
Epoch 30/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3111 - duration_loss: 0.0881 - pitch_loss: 3.7029 - step_loss: 0.0379
Epoch 31/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3153 - duration_loss: 0.0891 - pitch_loss: 3.7074 - step_loss: 0.0408
Epoch 32/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3172 - duration_loss: 0.0900 - pitch_loss: 3.7353 - step_loss: 0.0405
Epoch 33/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3369 - duration_loss: 0.0945 - pitch_loss: 3.8161 - step_loss: 0.0517
Epoch 34/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3289 - duration_loss: 0.0888 - pitch_loss: 3.7445 - step_loss: 0.0529
Epoch 35/50
506/506 [==============================] - 2s 5ms/step - loss: 0.3145 - duration_loss: 0.0846 - pitch_loss: 3.7044 - step_loss: 0.0447
Restoring model weights from the end of the best epoch.
Epoch 00035: early stopping
CPU times: user 1min 54s, sys: 23.9 s, total: 2min 18s
Wall time: 1min 23s
plt.plot(history.epoch, history.history['loss'], label='total loss')
plt.show()

png

Generar notas

Para usar el modelo para generar notas, primero deberá proporcionar una secuencia inicial de notas. La siguiente función genera una nota a partir de una secuencia de notas.

Para el tono de la nota, extrae una muestra de la distribución softmax de las notas producidas por el modelo, y no elige simplemente la nota con la probabilidad más alta. Elegir siempre la nota con la mayor probabilidad daría lugar a la generación de secuencias repetitivas de notas.

El parámetro de temperature se puede utilizar para controlar la aleatoriedad de las notas generadas. Puede encontrar más detalles sobre la temperatura en Generación de texto con un RNN .

def predict_next_note(
    notes: np.ndarray, 
    keras_model: tf.keras.Model, 
    temperature: float = 1.0) -> int:
  """Generates a note IDs using a trained sequence model."""

  assert temperature > 0

  # Add batch dimension
  inputs = tf.expand_dims(notes, 0)

  predictions = model.predict(inputs)
  pitch_logits = predictions['pitch']
  step = predictions['step']
  duration = predictions['duration']

  pitch_logits /= temperature
  pitch = tf.random.categorical(pitch_logits, num_samples=1)
  pitch = tf.squeeze(pitch, axis=-1)
  duration = tf.squeeze(duration, axis=-1)
  step = tf.squeeze(step, axis=-1)

  # `step` and `duration` values should be non-negative
  step = tf.maximum(0, step)
  duration = tf.maximum(0, duration)

  return int(pitch), float(step), float(duration)

Ahora genera algunas notas. Puedes jugar con la temperatura y la secuencia de inicio en next_notes y ver qué sucede.

temperature = 2.0
num_predictions = 120

sample_notes = np.stack([raw_notes[key] for key in key_order], axis=1)

# The initial sequence of notes; pitch is normalized similar to training
# sequences
input_notes = (
    sample_notes[:seq_length] / np.array([vocab_size, 1, 1]))

generated_notes = []
prev_start = 0
for _ in range(num_predictions):
  pitch, step, duration = predict_next_note(input_notes, model, temperature)
  start = prev_start + step
  end = start + duration
  input_note = (pitch, step, duration)
  generated_notes.append((*input_note, start, end))
  input_notes = np.delete(input_notes, 0, axis=0)
  input_notes = np.append(input_notes, np.expand_dims(input_note, 0), axis=0)
  prev_start = start

generated_notes = pd.DataFrame(
    generated_notes, columns=(*key_order, 'start', 'end'))
generated_notes.head(10)
out_file = 'output.mid'
out_pm = notes_to_midi(
    generated_notes, out_file=out_file, instrument_name=instrument_name)
display_audio(out_pm)

También puede descargar el archivo de audio agregando las dos líneas a continuación:

from google.colab import files
files.download(out_file)

Visualiza las notas generadas.

plot_piano_roll(generated_notes)

png

Consulta las distribuciones de pitch , step y duration .

plot_distributions(generated_notes)

png

En los gráficos anteriores, notará el cambio en la distribución de las variables de nota. Dado que existe un ciclo de retroalimentación entre las salidas y las entradas del modelo, el modelo tiende a generar secuencias similares de salidas para reducir la pérdida. Esto es particularmente relevante para el step y la duration , que utiliza la pérdida de MSE. Para pitch , puede aumentar la aleatoriedad aumentando la temperature en predict_next_note .

Próximos pasos

Este tutorial demostró la mecánica del uso de un RNN para generar secuencias de notas a partir de un conjunto de datos de archivos MIDI. Para obtener más información, puede visitar la generación de texto estrechamente relacionada con un tutorial de RNN , que contiene diagramas y explicaciones adicionales.

Una alternativa al uso de RNN para la generación de música es el uso de GAN. En lugar de generar audio, un enfoque basado en GAN puede generar una secuencia completa en paralelo. El equipo de Magenta ha realizado un trabajo impresionante en este enfoque con GANSynth . También puede encontrar muchos proyectos maravillosos de música y arte y código abierto en el sitio web del proyecto Magenta .