Author here. I’m a professional musician of 15 years transitioning into software development.
With this app I aimed to create an intuitive and usable drum sequencer that would run in the browser . I’ve always wanted to be able to program beats longer than 16 notes and see each instrument playing at the same time.
It’s built using React, Redux, and a fabulous web audio library called Tone.js.
This is the capstone to my learning to code which started in sept 2020. I was thrilled to be able to build something that combined my history in drumming and music production.
Happy to answer any questions!
EDIT: Link sharing was not working right but has been resolved. Share your beats! (You will have to re-save to generate a new url)
As a professional software engineer of ~20 years who's dabbled in music for as long as you've dabbled in software, I'm humbled and impressed. Just wanted to say "great job!", and wish you a long, fulfilling career combining your passions and talents.
Wonderful work! For a software engineer new to the space, your attention to detail and clarity of the implementation truly stand out.
Are you planning to open source and/or continue adding features to this app?
FYI, in case this wasn't your intent (I only mention this because you said you were fairly new up top), while you've provided the source you haven't given anyone a license to edit/duplicate/branch it.
Open Source usually implies a license, e.g. (reductively) the MIT ("do whatever you want as long as you credit me"), GPL ("If you use this code, you also have to share your code"), etc.
I actually made the same transition about 8 years ago. Coursera filmed this interview about my story (obviously cut down to mostly quotes praising them):
I was in basically this exact same spot a few years ago, and built something similar, and showing it off was a big part of my interview process where I got the job that changed my life/career. Wishing you the best. Let me know if there's any way I could help
There is! I’m coming from fifteen years in an unrelated (and wildly different) field. I am actively seeking advice related to landing a job where I would be a good fit. Please email me if you’d like to chat drumnickydrum@gmail.com
lol I built a drum machine too, its a nightmare of bash, perl, c, bc and invokations of sox. in my case, I can't show it off because the code is so bad it would do more harm than good.
I wrapped the entire mess up into an easily called script, and I love it despite the horrors of its internals.
I am a professional SWE and am exploring several music-oriented startup ideas at the moment. I definitely do not promise anything, but would love to stay in touch in case I get funded in the (distant) future. I will message you on GitHub.
I think it’s well designed, and I really like the analog drum samples (though I think the clap sounds better at G2 instead of C2). I would had made it 4 rows of 16 steps instead of 8 rows of 8 steps, but that’s because I’ve been using Roland grids for so long, that’s what I’m used to working with.
I also wish I could save a pattern as a long link. (MIDI slave sync support would also be nice, but I’m not sure browser sound libraries support that)
Thx! Menu dive a bit and you can re-pitch the clap. There is some, though not robust, MIDI support I will eventually be looking into. What do you mean exactly about saving a pattern as a long link?
Wow that site is super fun :) As far as the sequencer, head over to the save menu. If you 'share' your sequence it actually generates a link you can save. (It's not storing the data in the link as the full sequence data is too long at once for a url string.)
Another option is to use base-64 or XML (or even JSON) to store the data in a longish text string that’s still short enough to share online. For example, he’s a really nice ambient “bloom” reverb I use with Valhalla Supermassive (after downloading this free plugin, one can copy the following code then select “paste from clipboard” in Supermassive to load this reverb):
Sigh. Ever read an early proposal and think "wow, that's such a bad idea - good thing no one will pick up that"? Very not safe - a young naive me thought that reading early CGI, with its short urls passed on command-line, rather than longer urls on stdin. But people lycosed, and found a page, and were guided by it, and it spread, and... became assumed. Oh well.
Here in the future, with IE gone, IIRC browsers permit 80k, bottlenecked on Safari. Base-64 drops thats towards 60k. Of compressed data. Bzip can do a lot, even with JSON. Especially if you help it out.
There's some robustness cost, with unhappy security gateways and such, but for apps where featureful trumps reliable, we're no longer quite so crippled. Yay.
I believe this is fixed but can't test on my end. Anyone using old version of Safari that doesn't support MediaRecorder (or disabled MediaRecorder), please load the app and let me know if it goes blank.
ah, it finally let me respond to this comment, sometimes hn is sooo weird :)
safari 14.1 in Mojave. I spend too much time with weird audio stuff to move past Mojave on this laptop unfortunately.
edit:
aha -> looks like you have to turn on the experimental features to get it to work. might be good to have a note for someone who doesn't have that turned on to avoid the black screen.
Been looking into it. Looks like it's an old safari issue related to Media Recorder (for downloading your sequence.) I'll play around to make sure the ux is right regarding this bug.
Duuuude the mixer effects (reverb, filter, distortion, etc) are super responsive and wonderfully inspiring to play with — I haven’t even changed the pattern and had minutes of fun just playing with multiple effects at once, using two fingers to rhythmically move two sliders at the same time… this is just hilariously well done considering it’s all in-browser.
I hope you can keep working in the music space; this level of polish is sorely needed. Fantastic job, drumnickydrum; and thank you for sharing!
Running Firefox on Windows 10 x64, on a somewhat under-specced machine that tends to run low on RAM.
I'm getting highly inconsistent note timings, with intermittent full 1-second pauses with no new notes being triggered. (FWIW, I also saw something similar in https://news.ycombinator.com/item?id=26870666 , and am getting choppy audio in Zoom that disappears as soon as I enable profiling...)
Is it my browser's fault for having the JS event loop stall, or the page's fault for driving audio off the page's event loop? Or both? (IDK how to fix Firefox, other than increasing the content processes above 3, which will reduce cross-tab lag but possibly increase RAM, or to reduce my extensions or find the one at fault.)
On a related note, how critical is it to follow the "audio on a real-time thread" dogma (http://www.rossbencina.com/code/real-time-audio-programming-...)? I've heard that not using Electron, writing real-time code, avoiding mutexes and allocations, mlock to avoid swapping, tuning OSes and picking processors and audio interfaces based on its low-latency properties, etc. is important for live performances, but not necessary for MIDI instrument input for (casual home users? studio work?).
I can reproduce the audio stutter by having Firefox do anything performance intensive (starting recording a performance profile does the trick nicely) and I seem to be getting severe frame drops because of garbage collection.
I think the flows of React and Redux are just not built for timing-sensitive applications like these. A lot of allocations and deallocations are taking place without any user input. Looking at the source code, the many selectors and other hooks probably has something to do with that.
Redux is also very explicit about treating objects as immutable, causing a lot of duplication as a result. For a simple web application this shouldn't be too much of a problem but again, this application is very sensitive to timing issues where 50ms of GC can cause a noticeable performance degradation.
It's not that the code is bad, it seems quite well memo-ised to get decent performance; I think it's a result of using the wrong too for the job, nothing more. Quite impressive that a framework as heavy as React can be made this responsive in my opinion.
If you run into any memory troubles, check about:performance and about:memory, you may be able to find what's causing the memory leaks.
In the past I've found about:memory to be almost useless for pinpointing leaks. It's segmented into processes and types of allocations, and only the deeply nested sub-entries say which tab the memory belongs to. And even then, I don't know whether Firefox, the site, or an extension is causing memory bloat. I end up resorting to just blindly closing tabs or restarting the browser.
I might try about:performance more, but in the past I recall it being similarly almost useless for pinpointing sites eating CPU.
How many other open tabs and applications? Without a doubt I don't expect excessive overhead from the browser and would encourage users experiencing serious performance issues to wipe out everything else going on. This is still an experiment in just how much we can do in the browser.
It would be a fun challenge for me to programmatically figure out if the user is experiencing issues and disabled features to improve performance...
I work on the Web Audio API at Mozilla, here is an article that I wrote that explains what goes wrong here, distilled to the core: implementing a metronome in literate programming in a way that will _never_ glitch: https://blog.paul.cx/post/metronome/. The result is nowhere as interesting as this app, but this is to get the point across.
In 2021, with the AudioWorklet [0] being available in all major browsers (including Safari as of a few weeks ago!), arbitrary computations can happen on the real-time audio thread. It is now possible to send a copy of the sequencer data (which note when, what sample, what velocity, etc.) to the real-time thread, and let it play, unaffected by the load of the machine, be it from the browser or other programs (until everything breaks down because you're also overloading the audio thread).
I think this app is very good and does the best it can under the programming model it's using, considering the environment it's being run in, but if more robustness is needed, there is now a way !
On your last paragraph, it is still _absolutely_ critical. An audio software that doesn't use a real-time thread for rendering cannot render audio without glitches, even under no load, even on a fast machine. You don't need to tune the OS [1] and pick special hardware these days: today's computers (and even computers from 10 years ago or older) are perfectly capable or running reasonably complex audio processing workloads _if the software is written correctly_. This is regardless of the use case. When we disabled the real-time thread on Windows by mistake in Firefox (adjusting priority of the process level, and then that was inherited to all threads, very quickly reverted), we had reports in a matter of _days_ (if not the next day), and the Firefox Nightly population is not that big. This was I think playing regular videos on social media, not even anything complicated.
If the author of the app, or anybody, wants to profile the app with the right tools, I've written a blog post [2] about this as well. I hear this tooling is used by audio professionals, now, and that they're happy with it. Happy to hear about any suggestion though :-).
For your Firefox-specific question, you're right that increasing the content process count will make everything better, and we're constantly improving memory usage so that it's possible to do so even on lower-specs machines, maybe it's worth a try. Please look at the "resident memory" values, and not, say, virtual or shared, because we make heavy use of shared memory and copy on write facilities of the OS to limit the memory usage when using multiple process (which we need to do for origin isolation and avoiding Spectre issues and sandboxing various bits of the browser as hard as we can, for example).
[1]: (but you can to improve robustness further, and it helps to have good drivers, such as Jack on Linux, WASAPI with MMCSS on Windows or ASIO, or just the normal stuff on macOS)
Thanks for the post and your blog! Added to my RSS reader.
On the topic of MMCSS, RtAudio is another real-time audio API... except the WASAPI backend had broken real-time thread scheduling (AvSetMmThreadCharacteristicsW, I think this is MMCSS?) on any builds with UNICODE enabled... and somehow literally nobody noticed, until I analyzed RtAudio under Process Monitor and wondered why it was trying to open a random Chinese string as a DLL file. I ended up fixing it at https://github.com/thestk/rtaudio/pull/292/files .
I don't know what that means. Is MMCSS not that important (even though you mentioned missing MMCSS caused severe stuttering in Firefox)? Is RtAudio under-tested/amateurish to the point people didn't notice it was stuttering more than it should be?
I assume it's unacceptable to synthesize (or probably time) audio off the main thread. How bad is it to use mutexes and allocations on the audio thread?
I'm not sure how many Windows users RtAudio has, and the kind of machines people are using with it, and their expectations, but in Firefox, the answer is hundreds of millions and "any kind of machine that has existed", such as single or dual-core machines that are running dozens of modern web apps in parallel, so load is high. That might well explain why it was reported so quickly. It's quite noticeable with a standard load testing procedures (e.g. "how many parallel oscillators can this computer/OS/driver do at this latency and be glitch-less under load").
It is acceptable to do audio work off the main thread (as long as buffering is in place to prevent glitches from scheduling delays, and it's prioritized appropriately), but maybe you mean "on the main thread". In this case, it's not really recommended, but sometimes, it's what you need to do, for example when emulating an old console where everything is synchronous in the render loop.
Locks and allocations are best avoided, but modern locks and allocators are amazing. If the lock is _not contended_, it's very fast, and it's fine. If the allocation comes from a thread-local pool (some allocators do that), it's also very fast and fine, but that needs to be true across OSes, etc. for us. In other case, which is most of the time, it's best to avoid those.
We're not allocation nor lock-free entirely in Firefox though, but it's been thoroughly profiled - when know what we'll optimize when it's time to do so, but for now it's fine under extremely heavy load in a normal situation (we have locks when the audio devices are unplugged for an example of a "non-normal" situation). This is largely due to legacy (the Web Audio API, for example, is about 10 years old).
I experimented 5-6 years ago with a programmable metronome using KnockoutJS and realized quickly how easy it is to cause jitters and unpredictable performance on the main thread. I wish I had come across this post back then!
Although a big contributor for performance issues in my project (and especially OP's) is the amount of UI updates that need to happen in succession (lots of event listeners, css transformations and dom changes)
I'm working on a medium article as we speak. It's more about the journey from drumming to software, but does get a bit in depth with some of the challenges of S64.
Thankfully the timing is handled by the Tone.js framework. It allows you to sync events with a callback function which passes in a 'time' argument that keeps everything in line.
Very cool! On my underpowered HP Win10 machine running Firefox, turning on the Analyzer messes with the timing. Almost like introducing a random swing percentage, so potentially a feature rather than a bug. :-) Keep it up, I am a former professional bassist turned data analyst, always working on my programming chops.
I stayed up super late one night playing around with weird css combinations to make the analyzer do amazing things and ended up scrapping most of them to improve performance. It was a bit of an afterthought and I could certainly figure out a way to optimize it.
Great tool! I was a bit confused at first what was going on, especially when I clicked on an instrument. I think part of the disconnect is that the instruments are squares in a grid, but you see oval shapes in the sequencer itself. It may make more sense to show them as rounded rectangles in the sequencer so it's more obvious that each instrument maps to a colored shape in the sequence.
The interface is also begging to be used in both directions: it feels like I should be able to click on a square in the sequencer and see the instruments light up on the right, so that I can change multiple instruments at a time for a given square!
I hadn't thought about that direction of interaction. Very interesting. I agree, now that I think about it, that there are definitely a lot more interactions that can come off of the grid cells. Great suggestion.
Really fun! I couldn't figure out if there's a way to set a start and stop on the grid (so you could develop a beat in just 8 steps for example). Also can you set velocity/pitch for individual notes, not just all notes of a sample?
Thanks for the response, yeah the UI is a little tricky there. But it works OK when you figure it out. I really like painting the notes in on the grid, feels kind of different!
The share UI might need a UI tweak as well -- if you save locally without login first, then login, you see your track but can't AFAICT generate a share URL.
That's really sick, UI is really pretty! As an idea for a follow-up project, I wonder how difficult it would be to make it non-deterministic; i.e. play a different variation each measure.
I agree it would be fun to add a little chaos. I have some feature ideas on the todo list, one of which is a 'snake' traveler that disrupts the cells with an effect/glitch.
Thank you for the kind words. It means a lot. Many have mentioned I should sell this even though it wasn’t the intention. I’m not sure it will go that way but I’m stoked it lives up to the standard.
If you have your side button on mute you won't get any sound. If you don't mind, take a screen recording of the animation you're referring to and email it to me drumnickydrum@gmail.com
This is so much fun! The mixer effects sound great and I love the workflow, I've never seen this sort of nested-grid drum machine setup (instead of horizontal parallel tracks) but I find it really fun and intuitive.
I'm sure this is awesome so a a bit sad I'm blocked because it's not https. It's usuall trivial to set up https these days so that's my one and only feature request :)
The HN article link is the http url, that was the problem. If the http request is blocked, it never reaches the server to get redirected to the https version.
Is the only way to add a sound to first click the sound and then add it to squares? I would have preferred to click the square in the part of the 3x3 grid to add the sound directly.
Like falling in love with a new discipline much like when I first started playing drums. Complete (but healthy) obsession. And marrying the two together with the app kept me going for 10 weeks straight without getting bored with it. Excited to keep growing.
Thanks :) And not just for reusability of components... React's memoization features helped me prevent so much unnecessary re-rendering (the grid alone has about 1000 divs.)
Clever - You might be interested in Clojurescripts Reagent, which is a React wrapper that gives you a lot of memoization for free. Re-doing this in cljs would be a great learning exercise.
Oddly I'm finding there are much more audio performance issues in production even though all warnings pointed to worse in development.
It was an extreme challenge to make this thing work cross-browser/os/device. I may look into some performance diagnostics if playback isn't great for enough users. Thx for the report.
Dang. The download feature needs a lotttt more attention. I just felt wrong putting out the app without any ability to download what you've created. But without a doubt it needs more cross-platform support.
Thanks so much for it though. I can say it works well for me. Made an account and sharing it with my group of friends to make some terrible beats over :)
Thank you! I'm excited to see what's next, though I feel pulled in many directions. With native apps I can make real performance and UI gains, with web apps I can continue exploring the wild world of javascript (which I've grown to love.)
With this app I aimed to create an intuitive and usable drum sequencer that would run in the browser . I’ve always wanted to be able to program beats longer than 16 notes and see each instrument playing at the same time.
It’s built using React, Redux, and a fabulous web audio library called Tone.js.
This is the capstone to my learning to code which started in sept 2020. I was thrilled to be able to build something that combined my history in drumming and music production.
Happy to answer any questions!
EDIT: Link sharing was not working right but has been resolved. Share your beats! (You will have to re-save to generate a new url)
EDIT: Part I of the article is live: https://drumnickydrum.medium.com/from-funk-to-functions-da86...