Issuu

The evolving world of stem separation www.lalal.ai

Case Study

Splitting The evolving world of stem separation

nly a few years ago, extracting a convincing isolated vocal, drum part or bass line from a finished stereo mix seemed sufficiently improbable to qualify as some kind of technological miracle, but today, it is increasingly something you can do in almost every DAW. That change happened extraordinarily quickly, and obviously presents something of a challenge for a company like LALAL.AI whose business was initially built entirely around source separation. When your unique selling point has been performing a task that other music software can’t do, what happens when your customers are given the ability to carry out essentially the same process for themselves in the DAWs they are already using? Founded in 2020 by Russian entrepreneur Dmitry Orlov, the business of LALAL.AI is operated by Omnisale GmbH based in Holzhäusern, Switzerland, but unlike many software companies serving the music industry, LALAL.AI did not emerge from the traditional route of recording studios or digital audio processing. Its origins lie in a newer generation of what might be termed ‘AI-first’ businesses whose expertise lies far more in machine learning than audio signal processing.

LALAL.AI founder Dmitry Orlov is a Russian entrepreneur based in Germany with a background primarily in digital marketing and software business.

w w w. s o u n d o n s o u n d . c o m

Omnisale specializes in advanced machine learning and deep learning technologies tailored for audio signal processing, developed and refined entirely by their in-house research and development team. Access to such proprietary technologies is seen as one of the company’s competitive advantages.

Image The LALAL.AI system works in the frequency domain, using Short-Time Fourier Transform analysis to break the audio into time-frequency slices before neural processing. This hybrid approach is designed to maximise separation accuracy while keeping artefacts low. The models themselves are custom convolutional and recurrent networks, trained on large, hand-curated datasets and fine-tuned with loss functions that prioritise how the result sounds, not just how it measures. The models have also been made smaller and faster, optimised to run on GPU or TPU (Tensor Processing Unit: Google’s custom chip designed specifically for machine-learning) processors, so the same tech works both for end-users on the website and for other companies building it into their own apps and devices. LALAL.AI doesn’t train the separation models only on labelled examples such as ‘this is a vocal’; ‘this is a guitar’. They first give the neural network a large volume of ‘unlabelled’ music to listen to and analyse. The resulting ‘self-supervised’ learning means that the model is able to understand the general patterns of how music is

Omnisale Managing Director Céline Inglin.

Self-supervised learning means that the model is able to understand the general patterns of how music is structured. w w w. s o u n d o n s o u n d . c o m

There is nothing to adjust or set. You just have to decide what you want to separate out and check that the Preview works well enough. structured: what vocals sound like versus guitars, how rhythm works, and how different genres and languages are put together.

a centre-panned vocal by subtracting one stereo channel from the other is perhaps the simplest and most widely used early example.

Once equipped with that broader musical understanding, the model needs far less labelled data to then learn the specific job of stem separation. LALAL.AI trains its models on a large body of data specifically licensed and curated for model training. These datasets go through extensive cleaning and preparation before being used to develop and refine the models. New versions of models can be run side-by-side on real traffic to see which actually separates better in terms of speed, error rates, quality complaints and bleed, allowing for fast, small iterations.

But conventional audio signal processing doesn’t actually know what a vocal or a guitar is. It can only work on properties such as frequency, amplitude, stereo position, and perhaps harmonic structure to some degree, but instruments in a real mix overlap enormously in all of those dimensions. A snare drum alone contains significant energy across almost the entire frequency spectrum, a distorted guitar sound will occupy most of the same range as a vocal; and, of course, any reverb spreads sources across both time and the stereo space as well. Once those sources have all been combined in a mix, there often aren’t any clear boundaries between them.

And if all of that is a bit much to take in, rest assured that you don’t have to, because LALAL.AI’s processes have reduced all of that underlying complexity to the simplest user interfaces you’ll ever see!

The world before AI The concept of extracting individual sources from a mixed recording is much older than the current generation of AI tools. Engineers had been attempting versions of it for decades using phase cancellation, centre-channel extraction and filtering. The familiar trick of removing

w w w. s o u n d o n s o u n d . c o m

The advent of machine learning was able to change the approach completely. A neural network could be trained using enormous numbers of examples of individual sources and mixes, to allow the system to effectively ‘learn’ the characteristics that define and distinguish vocals, drums, bass and other instruments. And based on that knowledge, the process is able to make a sophisticated estimate of what each constituent source probably sounded like. A significant moment in the development of stem separation came with Deezer’s release of their open-source Spleeter system in 2019, which offered pre-trained models capable of separating recordings into two, four or five stems, including vocals, drums, bass, piano and the now familiar ‘other’ category. Spleeter was important not simply because

it worked, but because it made relatively sophisticated source separation widely accessible, with Spleeter’s technology soon being incorporated into products from other big brands. It might be said that the general direction of travel for this technology could already be seen in products like iZotope’s Music Rebalance in RX 7, launched in 2018, which allowed vocals, bass, percussion and other musical material to be independently adjusted in the mix, with some degree of success. The rapid development of increasingly capable neural-network architectures, however, embodied by systems such as the GitHub-hosted Demucs, pushed separation quality considerably further. A technology that had previously produced only highly compromised, artefact-laden audio was now capable of producing stems that could be genuinely useful for remixing, restoration, sampling and music production. There were plenty of artefacts still audible in isolation, but that wasn’t necessarily an issue for most users: a vocal extracted for inclusion in another arrangement won’t ever be heard in isolation: it only needs to be clean enough to work in its new context, and that rather resets the practical threshold for a successful stem split.

A specialist service This was environment into which LALAL.AI launched in 2020. Its founders believed that modern neural networks could deliver substantially better results if trained on larger data sets. According to the company, its first-generation network, known as Rocknet, was trained using approximately 20TB (twenty terabytes: around 20 trillion bytes, roughly 300,000 hours of WAV files, or a few million songs) of audio data, allowing it to outperform many contemporary solutions in preserving audio quality while reducing unwanted artefacts. Of course, the computing complexity involved in this was beyond the capabilities of most users own computer systems at the time, but a cloud-based service could run much more sophisticated models on powerful server hardware. The user only had to upload their audio and choose their desired stem separations, downloading the finished file once processing was complete. A Preview segment allowed you to check that the split was going to work before committing to processing the whole file and using up the ‘credits’ created by the subscription payment structure. As a user, you didn’t have to know anything other than what you wanted to split and whether or not what you were hearing in the Preview was good enough. There was nothing to adjust, effectively turning what was then still a relatively exotic technology into a simple consumer service.

LALAL.AI’s first product concentrated exclusively on the classic two-stem outcome, just separating everything vocal from ‘accompaniment’, or everything that wasn’t vocals. Although modest by today’s stem-splitting standards, it addressed the most common request from users at the time, allowing Karaoke creators to easily produce backing tracks, remix producers to isolate vocals for new arrangements and indeed anyone to be able to listen to a solo-ed vocal performance from a commercially released recording for the first time. The company’s rapid growth reflected the increasing maturity of deep-learning approaches to source separation across the industry and they followed up by embarking upon an ambitious programme of their own neural network development. Rocknet was succeeded by Cassiopeia, which significantly reduced processing artefacts while also enabling extraction of some individual instruments rather than simply vocals and accompaniment. During 2021 LALAL.AI’s stem separator expanded into what the company described as the world’s first commercially available eight-stem separator, capable of isolating drums, bass, piano, synthesisers, acoustic guitar and electric guitar, alongside vocals and other instrumental backing. This helped distinguish LALAL.AI from the many competitors that had since sprung up, particularly the free or low-cost utilities whose functionality remained largely limited to vocal extraction. 2022 saw the arrival of the Phoenix neural network, which became the foundation for LALAL.AI’s next generation of stem separation, offering further improved isolation quality while expanding support to more instrument families such as strings

The People’s Voice

In 2026 LALAL.AI won the ‘People’s Voice Award for Best Use of AI & Machine Learning’ in the Apps, Software & Immersive category at the 30th Annual Webby Awards, an accolade hailed by The New York Times as “the Internet’s highest honor.” The ‘People’s Voice Award’ is decided entirely by public vote, with millions of ballots cast each year by internet users around the world. LALAL.AI’s win came against a shortlist of prominent AI products, including Adobe Premiere Object Mask, Beeble AI, Captions AI, and Cosmos. In addition to the ‘People’s Voice’ win, LALAL.AI also received a ‘Webby Honoree’ distinction in the Creator, Creative, and Media Tools category, also under Apps, Software & Immersive, recognizing the platform among the top entries in that category as well.

w w w. s o u n d o n s o u n d . c o m

and wind instruments. These additions reflected an increasingly sophisticated understanding of complex orchestral and acoustic material, seeking to extend the platform’s usefulness beyond mainstream popular music.

Wider applications During the same period, the company introduced Voice Cleaner and Echo & Reverb Remover — AI-powered restoration tools capable of removing unwanted background noise, microphone handling noise, plosives and other forms of environmental contamination. These tools were especially useful for a new generation of small-scale podcast, online video production and live-streaming creators, often working in poor recording environments, allowing them to achieve results that previously may have required a purpose-built studio room. This marked LALAL.AI’s first significant move into the broader field of intelligent audio enhancement rather than just musical stem extraction.

The real ‘win’ now lies in being able to tackle the really difficult splits, like piano from guitar, electric guitars from acoustics, and separating out lead and backing vocals. Wider accessibility has also played a central role in the company’s expansion. What began as a browser-based service gradually evolved into desktop applications for Windows, macOS and Linux, followed by dedicated iOS and Android apps. More recently, developer APIs have allowed third-party software vendors to integrate LALAL.AI’s separation technology into their own products and workflows, extending the platform beyond individual end users. Another notable aspect of LALAL.AI’s development has been its continuing investment in proprietary neural architectures. After Phoenix came Orion, which the company describes as combining source separation with enhancement, attempting not just to isolate stems but also to reduce residual imperfections during processing. Subsequently, Perseus and Andromeda models have become the foundation of the latest

w w w. s o u n d o n s o u n d . c o m

generation of services. Andromeda is said to have been trained using roughly four times more data than Perseus, offering faster processing alongside measurable improvements in separation quality and reduced crosstalk.

Voice transformation Today, the product range extends still further, into the field of generative AI voice technology. LALAL’s Voice Changer app enables users to transform recorded vocals into a range of alternative vocal identities, while Voice Cloner allows the creation of reusable digital versions of the user’s own voice from uploaded speech samples for applications including audiobooks, advertising or narration. Voice cloning and voice conversion can also support multilingual localisation, automated customer communications and accessibility services for individuals who have lost their natural speaking voice through illness or injury. When used with appropriate consent and governance, synthetic voices can preserve a person’s vocal identity for future communication, allowing them to continue speaking in a recognisable version of their own voice through assistive technologies. Educational and research environments represent another growing area of interest, with music educators now able to use stem separation to isolate individual instruments for demonstration purposes, allowing students to more easily analyse and study bass lines, drum parts, orchestration or vocal harmonies in commercially released recordings. Musicologists similarly gain new opportunities to analyse performance practice, arranging techniques and instrumental interaction within historical recordings that predate multitrack recording. For students learning production, being able to deconstruct a finished mix into its constituent elements provides an invaluable insight into balance, arrangement and recording techniques. In the fields of film, television and post-production, source separation has become an increasingly useful restoration tool. Archive material often exists only as mixed mono or stereo recordings in which dialogue competes with music and effects. While AI cannot recreate missing production stems, it can frequently produce cleaner dialogue tracks for remastering, subtitling, dubbing or accessibility purposes. Similar techniques can also assist forensic audio specialists tasked with improving the intelligibility of surveillance recordings, emergency calls or historical archive material, although the limitations and evidential implications of AI-enhanced recordings must always be carefully understood and documented. Finally, LALAL.AI’s cloud-based architecture and API services make its technology attractive as an enabling component within larger software ecosystems. Rather than interacting directly with the LALAL.AI website, broadcasters, media asset management systems, transcription platforms and content management applications can integrate source separation and voice enhancement into automated production pipelines. In this context, AI separation becomes less a standalone creative tool and more an integrated stage within a wider workflow, improving the quality of material before transcription, translation, indexing, or downstream machine-learning analysis. The

ability of LALAL.AI’s technology to function as underlying infrastructure may ultimately prove just as significant for the business as its more visible role in direct end-user content processing.

LALAL.AI plug-ins LALAL.AI has also just launched some of their processes as plug-ins, including the Stem Separator, Speech Cleaner and Voice Dereverb. Unsurprisingly, the Stem Separator, isn’t the company’s high-powered cloud service wrapped in a conventional plug-in interface, giving access to their best-quality stem separation without leaving your DAW. For now, at least, LALAL.AI’s flagship Andromeda, Perseus and Orion cloud architectures are clearly beyond the capabilities of even the most powerful of users’ own computers, so the plug-in has been created around a new neural network designated Lyra. Whilst a cloud-based service can exploit enormous computational resources, abundant memory, and processing times measured in terms of ‘as long as it takes’, a plug-in operating inside a DAW must operate in near real-time, with a consistent

LALAL’s Voice Changer app enables users to transform recorded vocals or speech into a range of alternative vocal identities. latency that can accurately be reported to the latency compensation engine, and also coexist with dozens of other plug-ins. Unsurprisingly, Lyra doesn’t claim to achieve the same audio quality as LALAL.AI’s cloud-based processes.

w w w. s o u n d o n s o u n d . c o m

Many DAWs now include a non-real-time stem splitter in their integrated plug-in bundles, operating entirely locally, and offering, in general, acceptable results, at least on the ‘easier’ stem sources, such as drums and vocals. The potential of the plug-in, therefore, lies in the way in which it could eliminate altogether the interruption of having to process the audio as a separate action. The latency behaviour, however, is strongly dependent on the DAW host and its plugin delay-compensation limits. In some hosts, the current model’s processing latency can exceed the amount of delay the host is able to compensate for, which will affect synchronization. You can still use a non-real-time bounce or render, although the exact synchronization behaviour will depend on how the DAW handles plugin latency and delay compensation. Lyra is designed to provide strong local separation within the computational constraints of a DAW, but as with any stem-separation system, results will always remain highly dependent on the source material. All of which means that LALAL.AI plug-in doesn’t quite do what we would all like it to do. At least not yet. Which is a shame as there are numerous super useful applications for a real-time splitter that will operate in sync with a DAW timeline: want to lift the drums in a mix when you haven’t got access to the multitrack? Just run two tracks of the mix in parallel and stem split the drums in the second one. Now you have a channel of drums to boost as required. Wish there was a hint more reverb on the snare? EQ the top and bottom out of the drum stem and add a touch of reverb to what remains. Subtly blended back in, the snare will have a touch more reverb. The same goes for vocals, or any other source the splitter can peel off. Or you could just use the stem-extracted audio for control signals: a drum track could be used to drive a compressor side-chain or envelope shaper; an extracted guitar could trigger ducking; a bass extraction could control a dynamic EQ. And if the stem is just being used as a side-chain detector rather than audio that will be heard directly, artefacts in the extraction are much less important. Of course, you can do all of these things with your DAW’s offline plug-in, but the great thing about being able to hear it in real-time is that it will quickly tell you if it is going to work, and whether it is worth subsequently spending the time doing it with a higher quality processor. These limitations reflect the current generation of the local model rather than the direction of the technology. LALAL.AI is already working on lower-latency solutions that are planned for future releases, with substantially lower computational requirements. The aim is to make real-time, DAW-synchronised stem separation increasingly responsive and practical across a wider range of systems.

Lower-latency solutions with substantially lower computational requirements are planned for future releases, seeking to make real-time, DAW-synchronised stem separation practical across a wider range of systems.

w w w. s o u n d o n s o u n d . c o m

The LALAL.AI Stem Splitter plug-in is presently available only as a VST3 format, with an AU version in beta — Pro Tools users don’t have native AAX compatibility yet, limiting the plug-in’s appeal within a significant part of the wider commercial market. LALAL.AI states that Lyra supports acceleration using modern graphics processors, so, as neural processing units become increasingly common in desktop and laptop computers, the local-processing approach may prove indicative of a broader shift throughout the audio software industry, with more machine-learning tasks migrating away from cloud services and onto dedicated hardware within the studio computer itself. It may be not unreasonable therefore to expect to see processes

like LALAL.AI’s plug-in able to work to their full potential in the near future, perhaps as an ARA2 process (Audio Realtime Access) that utilises a certain amount of pre-processing. There is another aspect to local processing, of course, and that is the issue of privacy and confidentiality. If your business involves handling unreleased commercial works, or valuable back-catalogue recordings, or even developing advertising campaigns, then uploading that material to cloud servers may not always seem like a safe option. A workflow in which all processing remains local, with the audio never having to leave the workstation, is clearly preferable.

Market Positioning With basic stem separation having migrated into a staple process of mainstream music software, LALAL.AI responded by increasing the granularity of their separation, expanding to ten categories: vocals, instrumental, drums, bass, piano, acoustic guitar, electric guitar, synthesizer, strings and wind instruments. Whilst simply wanting to extract a vocal may be perfectly adequate for many users, anyone wanting reconstruct an arrangement, extract samples or perform restoration work, being able to separate out more sources is essential. The real ‘win’ now lies in being able to tackle the really difficult splits, like piano from guitar, electric guitars from acoustics, and separating out lead and backing vocals. Whether this is deeper than the DAWs will ultimately want to go remains to be seen.

The other, and arguably most important, key differentiator, though, is quality, both in terms of the correct identification of the chosen sources and the usability of the resultant audio in real work. You might imagine that your paid-for, cloud-processed separations are always going to be better than the ones you can get with the integrated splitter in your DAW, but anyone with personal experience of a wide range of splitters knows that is not necessarily the case. The process by its very nature is always going to be highly dependent on the source material it is presented with, and even the best splitters are capable of producing softened transients, alien vocals, bass-notes with no attack or unwanted modulation, and the characteristic, all too familiar, sound of partially removed spill

Not all the trained models are the same, of course, so it is no surprise that the same source fed into two different splitters can sometimes produce really different results. from other sources in the background. But not all the trained models are the same, of course, so it is no surprise that the same source fed into two different splitters can sometimes produce really different results. This has resulted in some users looking to specific providers for specific sources — unsurprisingly, UVR (Ultimate Vocal Remover) is highly rated for vocals, and I and many others have evolved a preference for the Moises splitter for guitar tracks, particularly for its ability

Neko — stem separation and more Another direction for LALAL.AI is bringing stem separation into hardware designed for musicians. The company has started working with UAE-based Neko Engineering on a compact AI music device for guitarists called Neko, moving LALAL.AI’s technology beyond the cloud and users’ computers to where musicians start the creative process. In addition to stem splitting imported tracks, Neko runs NAM A2 amp and speaker models and connects directly to TONE3000, the world’s largest online community for creating, sharing and discovering Neural Amp Modeler captures and impulse responses. Guitar connection can be wired, via standard quarter-inch jack, or wireless (with 3ms latency) and Neko incorporates an AI arranger — record some audio and

Neko can arrange it into a finished song with drums, bass and more. Neko will also convert guitar stems into readable tabs — great for learning parts. or just playing along with original tracks.

Neko is expected to retail for $499, although early adopters can currently pre-order the device for $349, with deliveries scheduled to begin in 2027.

w w w. s o u n d o n s o u n d . c o m

It’s no surprise to see vocals as the most frequently used stem split. to separate electric and acoustic guitars within a mixed stem. But there are still times when LALAL.AI will provide the best outcome on those sources when the favoured separator just isn’t quite doing the job. LALAL.AI meanwhile has continued developing increasingly sophisticated cloud models, with their current flagship Andromeda having been trained on four times as much data as its preceding Perseus model. This helps it to not only process audio faster, but also achieve a 10-percent improvement in SDR (Signal-to-Distortion Ratio), which in the context of stem separation, is a commonly used objective measure of how closely an extracted stem resembles the original, clean source. There should, for a while at least, remain some distinction between the overall capabilities of a cloud-based specialist and anything we are going to find in a DAW, where manufacturers are going to have to keep

w w w. s o u n d o n s o u n d . c o m

their models relatively small and fast — nobody wants to trigger a DAW editing command that ends up occupying their computer for twenty minutes! Also, whilst this may seem trivial compared with neural-network architectures, the user interface really does matter, especially outside of mainstream, serious music production. One of LALAL.AI’s original strengths was partly that the user didn’t need to understand source separation at all, and the current service preserves that simplicity: you choose a file, choose what you want separated and process it. And that potentially

More advanced generative models could increasingly rebuild damaged or obscured components rather than merely extracting them.

reaches a much larger market than DAW users. Someone making a karaoke backing track doesn’t necessarily own Ableton Live; a guitarist trying to remove guitar from a song for practice may have no interest in learning Logic; a video editor wanting cleaner dialogue doesn’t necessarily think in terms of musical integrity or neural source separation. For those customers, the absence of complexity is some of the product’s appeal. Perhaps this is not unlike what has happened with audio mastering. Sophisticated mastering tools have existed within DAWs for decades, but that didn’t prevent automated mastering services from finding customers among DAW users. However good we make the tools, capability and the ability to achieve the outcome you want aren’t the same thing.

Where does stem separation go next? LALAL.AI’s competitive future may not rest just on their present combination of audio quality, stem granularity, accessibility, platform independence, batch processing, cloud computing power, local/ private processing and API integration. There is also a surprisingly important advantage in being ‘DAW-agnostic’. Logic’s decent Stem Splitter is of little help to a Windows user, and some integrated separators require particular hardware or particular editions of the software, such as Logic’s Stem Splitter requiring Apple silicon rather than Intel Macs. LALAL.AI can also operate through a browser or dedicated applications on macOS, Windows and Linux, while its VST3 plug-in can be used in numerous hosts. But, inevitably, once everybody can produce a reasonably convincing vocal, drums, bass, other instruments split, specialist developers are increasingly competing in the arena of even more difficult challenges rather than the routine ones, and that’s somewhere we all want them to go. More detailed ‘decomposition’ is surely the destination: kick, snare, toms, cymbals and percussion, rather than simply ‘drums’; individual singers rather than ‘vocals’; rhythm and lead guitar rather than ‘guitar’; individual instruments within orchestral recordings rather than ‘strings’. Imagine having temporal and semantic understanding in which a system might be able to recognise that the guitar playing the intro and the guitar playing the solo are the same instrument and place them on one track, while identifying another overdub as a separate performance with a different musical function within the piece. There is also a progression from just separation towards reconstruction. Today’s systems predominantly estimate the contribution of different sources to an existing recording. More advanced generative models could increasingly rebuild damaged or obscured components rather than merely extracting them — potentially producing

something closer to a clean, editable approximation of the original multitrack. At that point, the distinction between source separation, audio restoration and generative audio begins to blur. The user doesn’t need to understand how it is done in order to be able to use the outcome. Stem separation is now just a basic enabling technology, rather like time-stretching, pitch correction or noise reduction, but the focus of the major AI players is already on the more interesting products yet to be built. Competition within AI audio processing has intensified dramatically in recent years, with commercial offerings from many new and established providers, as well as numerous open-source projects, but LALAL.AI may have a competitive edge through its ability to continuously refine its proprietary neural networks. The company is now one of the industry’s most comprehensive AI-powered audio platforms, reflecting the wider transformation of artificial intelligence from a specialist research discipline into an everyday production tool for creators at every level of the music and media industries. LALAL.AI’s growing popularity appears to support that philosophy with almost 69 million splits performed in a single year, representing almost 15 million hours of audio! The company reports that its registered user base grew from approximately 3.8 million during 2024 to nearly 6.8 million during 2025, suggesting that they remain well positioned to push the boundaries of this technology well into the future.

ACKNOWLEDGEMENTS www.lalal.ai Special thanks to: Maria Fedoseeva Text: Dave Lockwood.

w w w. s o u n d o n s o u n d . c o m