<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/layers/preprocessing/mel_spectrogram.py#L9-L380) |

A preprocessing layer to convert raw audio signals to Mel spectrograms.

Inherits From: [`Layer`](https://www.tensorflow.org/api_docs/python/tf/keras/Layer), [`Operation`](https://www.tensorflow.org/api_docs/python/tf/keras/Operation)

    tf.keras.layers.MelSpectrogram(
        fft_length=2048,
        sequence_stride=512,
        sequence_length=None,
        window='hann',
        sampling_rate=16000,
        num_mel_bins=128,
        min_freq=20.0,
        max_freq=None,
        power_to_db=True,
        top_db=80.0,
        mag_exp=2.0,
        min_power=1e-10,
        ref_power=1.0,
        **kwargs
    )

This layer takes `float32`/`float64` single or batched audio signal as
inputs and computes the Mel spectrogram using Short-Time Fourier Transform
and Mel scaling. The input should be a 1D (unbatched) or 2D (batched) tensor
representing audio signals. The output will be a 2D or 3D tensor
representing Mel spectrograms.

A spectrogram is an image-like representation that shows the frequency
spectrum of a signal over time. It uses x-axis to represent time, y-axis to
represent frequency, and each pixel to represent intensity.
Mel spectrograms are a special type of spectrogram that use the mel scale,
which approximates how humans perceive sound. They are commonly used in
speech and music processing tasks like speech recognition, speaker
identification, and music genre classification.

> [!NOTE]
> **Note:** This layer is safe to use inside a [`tf.data`](https://www.tensorflow.org/api_docs/python/tf/data) or `grain` pipeline (independently of which backend you're using).

#### References:

- [Spectrogram](https://en.wikipedia.org/wiki/Spectrogram),
- [Mel scale](https://en.wikipedia.org/wiki/Mel_scale).

<br />

<br />

<br />

| ## Args ||
|---|---|
| `fft_length` | Integer, size of the FFT window. |
| `sequence_stride` | Integer, number of samples between successive STFT columns. |
| `sequence_length` | Integer, size of the window used for applying `window` to each audio frame. If `None`, defaults to `fft_length`. |
| `window` | String, name of the window function to use. Available values are `"hann"` and `"hamming"`. If `window` is a tensor, it will be used directly as the window and its length must be `sequence_length`. If `window` is `None`, no windowing is used. Defaults to `"hann"`. |
| `sampling_rate` | Integer, sample rate of the input signal. |
| `num_mel_bins` | Integer, number of mel bins to generate. |
| `min_freq` | Float, minimum frequency of the mel bins. |
| `max_freq` | Float, maximum frequency of the mel bins. If `None`, defaults to `sampling_rate / 2`. |
| `power_to_db` | If True, convert the power spectrogram to decibels. |
| `top_db` | Float, minimum negative cut-off `max(10 * log10(S)) - top_db`. |
| `mag_exp` | Float, exponent for the magnitude spectrogram. 1 for magnitude, 2 for power, etc. Default is 2. |
| `ref_power` | Float, the power is scaled relative to it `10 * log10(S / ref_power)`. |
| `min_power` | Float, minimum value for power and `ref_power`. |

<br />

#### Examples:

**Unbatched audio signal**

    layer = keras.layers.MelSpectrogram(num_mel_bins=64,
                                        sampling_rate=8000,
                                        sequence_stride=256,
                                        fft_length=2048)
    layer(keras.random.uniform(shape=(16000,))).shape
    (64, 63)

**Batched audio signal**

    layer = keras.layers.MelSpectrogram(num_mel_bins=80,
                                        sampling_rate=8000,
                                        sequence_stride=128,
                                        fft_length=2048)
    layer(keras.random.uniform(shape=(2, 16000))).shape
    (2, 80, 125)

<br />

<br />

<br />

| ## Input shape ||
|---|---|
| 1D (unbatched) or 2D (batched) tensor with shape:`(..., samples)`. ||

<br />

<br />

<br />

<br />

| ## Output shape ||
|---|---|
| 2D (unbatched) or 3D (batched) tensor with shape:`(..., num_mel_bins, time)`. ||

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `input` | Retrieves the input tensor(s) of a symbolic operation. <br /> Only returns the tensor(s) corresponding to the *first time* the operation was called. |
| `output` | Retrieves the output tensor(s) of a layer. <br /> Only returns the tensor(s) corresponding to the *first time* the operation was called. |

<br />

## Methods

### `convert_weight`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/layers/preprocessing/data_layer.py#L153-L159)

    convert_weight(
        weight
    )

Convert the weight if it is from the a different backend.

### `from_config`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/ops/operation.py#L322-L362)

    @classmethod
    from_config(
        config
    )

Creates an operation from its config.

This method is the reverse of `get_config`, capable of instantiating the
same operation from the config dictionary.

> [!NOTE]
> **Note:** If you override this method, you might receive a serialized dtype config, which is a `dict`. You can deserialize it as follows:

    if "dtype" in config and isinstance(config["dtype"], dict):
        policy = dtype_policies.deserialize(config["dtype"])

<br />

<br />

<br />

| Args ||
|---|---|
| `config` | A Python dictionary, typically the output of `get_config`. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| An operation instance. ||

<br />

### `linear_to_mel_weight_matrix`

[View source](https://github.com/keras-team/keras/tree/v3.15.1/keras/src/layers/preprocessing/mel_spectrogram.py#L202-L335)

    linear_to_mel_weight_matrix(
        num_mel_bins=20,
        num_spectrogram_bins=129,
        sampling_rate=8000,
        lower_edge_hertz=125.0,
        upper_edge_hertz=3800.0,
        dtype='float32'
    )

Returns a matrix to warp linear scale spectrograms to the mel scale.

Returns a weight matrix that can be used to re-weight a tensor
containing `num_spectrogram_bins` linearly sampled frequency information
from `[0, sampling_rate / 2]` into `num_mel_bins` frequency information
from `[lower_edge_hertz, upper_edge_hertz]` on the mel scale.

This function follows the [Hidden Markov Model Toolkit (HTK)](http://htk.eng.cam.ac.uk/) convention, defining the mel scale in
terms of a frequency in hertz according to the following formula:

`mel(f) = 2595 * log10( 1 + f/700)`

In the returned matrix, all the triangles (filterbanks) have a peak
value of 1.0.

For example, the returned matrix `A` can be used to right-multiply a
spectrogram `S` of shape `[frames, num_spectrogram_bins]` of linear
scale spectrum values (e.g. STFT magnitudes) to generate a
"mel spectrogram" `M` of shape `[frames, num_mel_bins]`.

    # `S` has shape [frames, num_spectrogram_bins]
    # `M` has shape [frames, num_mel_bins]
    M = keras.ops.matmul(S, A)

The matrix can be used with [`keras.ops.tensordot`](https://www.tensorflow.org/api_docs/python/tf/keras/ops/tensordot) to convert an
arbitrary rank `Tensor` of linear-scale spectral bins into the
mel scale.

    # S has shape [..., num_spectrogram_bins].
    # M has shape [..., num_mel_bins].
    M = keras.ops.tensordot(S, A, 1)

#### References:

- [Mel scale (Wikipedia)](https://en.wikipedia.org/wiki/Mel_scale)

<br />

<br />

<br />

| Args ||
|---|---|
| `num_mel_bins` | Python int. How many bands in the resulting mel spectrum. |
| `num_spectrogram_bins` | An integer `Tensor`. How many bins there are in the source spectrogram data, which is understood to be `fft_size // 2 + 1`, i.e. the spectrogram only contains the nonredundant FFT bins. |
| `sampling_rate` | An integer or float `Tensor`. Samples per second of the input signal used to create the spectrogram. Used to figure out the frequencies corresponding to each spectrogram bin, which dictates how they are mapped into the mel scale. |
| `lower_edge_hertz` | Python float. Lower bound on the frequencies to be included in the mel spectrum. This corresponds to the lower edge of the lowest triangular band. |
| `upper_edge_hertz` | Python float. The desired top edge of the highest frequency band. |
| `dtype` | The `DType` of the result matrix. Must be a floating point type. |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A tensor of shape `[num_spectrogram_bins, num_mel_bins]`. ||

<br />