ASoC: meson: axg-tdm-interface: Fix formatters in trigger"

ASoC: meson: axg-tdm-interface: Fix formatters in trigger"

This reverts commit c97d3959035352ae99dde47303cea2d9e3c928d6 because
the following and required commit 1f4bba52eac28b8075038b558113bba6c2486bf7
causes the following system crash when using audio:
 BUG: sleeping function called from invalid context at kernel/locking/mutex.c:282

Fixes: c97d3959035352ae99dde47 ("ASoC: meson: axg-tdm-interface: manage formatters in trigger")
Reported-by: Dmitry Shmidt <dimitrysh@google.com>
Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
Acked-by: Jerome Brunet <jbrunet@baylibre.com>
Link: https://lore.kernel.org/r/20220421155725.2589089-1-narmstrong@baylibre.com
Signed-off-by: Mark Brown <broonie@kernel.org>