I think the thing that non-experimentalists don't realize is how many experiments fail for reasons that are not understood. For example - "the neurons in the part of the brain failed to express the transgene I wanted to use to study them." To actually write a paper about that null result requires substantially more experiments to form hypotheses and test them. Moreover, my lab is far from being an expert in the esoteric ways that this can happen, so when I try to submit the paper, the experts ask for even 10x more experiments. So I've gone farther and farther from the questions that are "important" (reflect my interests, expertise, and funding).

But this explains the value of scientific meetings and poster sessions where I can just tell this random tidbit to interested people without the burden of peer review.

An interesting recent development I've observed once was a tweet describing the fact that a dye (Texas red) labels brain vasculature even when injected subcutaneously. This random fact is not in the literature but is quite helpful from a procedural perspective. I think that science twitter has a potentially super valuable role to play in reporting unexpected or otherwise difficult to publish findings.

Most "failed" pilots don't demonstrate that an approach is fundamentally and irredeemably flawed, which could be an interesting paper.

Instead, they demonstrate that doing something this specific way is too much of a hassle, too unreliable, or too expensive to answer a particular question in a particular context: skills, budget, timeline, resources, alternative ideas, etc.

I don't see how you could write pilot results up ("Trust me, I'm usually pretty good at these sorts of things but this didn't work well"?). Meanwhile, disentangling these factors in a rigorous, generalizable way would turn it into an entirely different project.

I think that is a fair point. It is partially answered by whether a grant was received, but only partially.

On the other hand, data sharing protocols mean the data is going to be made available, so I think that probably addresses that issue.

The problem with pilot data is that lots of things can change in the course of running the pilot; tweaks of the experimental protocol, bug fixes to code, etc.

Yes. The details of the used protocols are supposed to be included (and usually are) in the primary publication of the research.

But not the results of the pilot unless it was promising?

pilot data is not what you think it is. Pilot data is about finding parameters, testing procedures. It is neither a consistent nor a full replica of an actual experiment.