As always, this is why peer-review is taken in such high regard. Replicate, replicate, replicate.
this post was submitted on 01 Aug 2023
179 points (100.0% liked)
Technology
39598 readers
322 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 3 years ago
MODERATORS
Well, just to push back a little on any impression some might get from this episode of the health of science (all IMO of course)
Most things aren't subjected to replication attempts like this, largely because I think people have a decent amount of self-interest in getting on top of this material as early as possible if the claims are real, and, the manufacturing of the material is relatively trivial. In science in general, game changing technologies or techniques can get replication attention like this, but overall a lot of "discoveries or findings" just aren't challenged as there is no real incentive to do so as a researcher, to the point that often you'll get pushback if you try to publish a failed replication study.
our compound shows greatly consistent x-ray diffraction spectrum with the previously reported structure data
Uhh, doesn’t look like it to me. This paper’s X-ray diffraction spectrum looks pretty noisy compared to the one from the original paper, with some clear additional/different peaks in certain regions. That could potentially affect the result. I was under the impression from the original paper that a subtle compression of the lattice structure was pretty important to formation of quantum wells for superconductivity, so if the X-ray diff isn’t spot on I’ll wait for some more failures before calling it busted.
yea interesting! It's definitely the arc I'm hoping for here ...
that either the material is tougher to make than the papers suggest, or,
to get into my fantasy land, the material they made is a superconductor but they don't really know why or how to make it the way they did as it was kinda some accident they weren't in control of. If true, it would make whatever is left of the material rather valuable and subject to some drama I'd imagine.
Wait, did people actually believe this was real? I’d seen it faked before, so was a bit jaded at the news.
Glad to have peer reviews!
I did not know this story! Thanks! Important precedent it seems in framing a foundation of scepticism.
Unfortunately it’s a 3 part (~2.5 hour) series, but I thought it was worth the time. Definitely made me wary on the topic LOL
I was, and am, skeptical, but I also must admit, the potential breakthrough is teasing my psyche with that feeling of just wanting it to be real. A part of me hopes that maybe it will still end up confirmed by other peers, but, granted, it was a low chance even when the news first came out.
From what I have heard, it's not supposed to be that expensive or even difficult to make. They should have sent actual samples of the material to a dozen different universities from a batch they share their own data measurements about. Save everyone a lot of time about doubts that it's manufactured correctly.
From that article:
“The general public seems oddly pumped about how ‘easy’ the 4-day, multistep, small batch, solid state synthesis is,”
The process is a 3-stage heating-holding-cooling process which they haven't published the precise temperature profile for. The papers also claim only 4 samples were ever made in total, 2 of them got (destructively) analyzed by gas spectroscopy and crystallography, while of the other 2, one got further temperature annealed, and both got electrically characterized.
Chances are, they themselves don't know exactly how they got what they got, and may or may not be capable of producing more samples.
I imagine they only have a few grams of the stuff. And they're not highly motivated to be debunked immediately
but if it were real, they were claiming it could be manufactured easily, so making some samples to send out to research labs would not be too difficult
Right. And they haven't. Now what may we deduce about their confidence in these results?
For anyone interested, there is a forum thread which is the closest thing we have to a live blog, along with the thread author's opinions on how veritable the claims of each party currently known to try and replicate the study are.
Would a mega thread here help?
Though, lemmy.world is still defederated right? sh.itjust.works too? Maybe it'd be nicer to have one they could access too?
Meh, I think the people that are interested in the spacebattles thread will have found it through other means like HN. As for discussing the events as they unfold I think that's best left to the forum itself, rather than discuss it at a proxy on Lemmy
I think it will be a while before we know what's really happened.
Something I find striking is the question of where their original material is and where's the video evidence of them testing it?
If I allow myself to be somewhat conspiratorial, I'd imagine that they know the material they made may have been somewhat accidental and that any further progress may depend on analyzing the material itself to determine what makes it work, which means they may want to keep its location somewhat secret.
Otherwise, I'm inclined to think that there's something funky going on within the dynamics of the research group and that not one of them is entirely on top of everything that happened with the material and so the evidence got mixed up and foggy.
It is notoriously hard to replicate things in labs, especially with material science.
This was attempt to do it within 2 days of the paper being published.
To add to that, the original researchers apparently had 10% successes rate in their lab, they wanted to perfect it before publishing their paper.
Bad luck was that it leaked, so to make sure somebody else doesn't get credit for their work they published what they had within hours.
It likely will take months before this will be verified.
10% success rate suggests there's some hidden factor they haven't discovered themselves yet, might influence the success rates of other labs. (assuming of course the claim is not fabricated)
That's often how it goes. Something doesn't do what you expect, so you have to keep trying new things until you figure out why it wasn't what you expect.
It's a shame that it so far seems that this superconductor experiment was a bust, but even still, I'm happy to see the scientific process at work.
load more comments
(5 replies)
There goes my hopes dashed
Ok, what is the importance of a room temperature semiconductor.
room temperature semiconductor
A room temperature superconductor would allow 100% efficiency for energy transmission and allow all sorts of technologies like cheap maglev trains using flux pinning for example.
You know how your phone and computer heat up when doing something intensive? This happens due to the resistance inside it. Superconductors would allow this electricity to pass through with virtually no resistance, generating no heat.
load more comments
(12 replies)
So is this confirmation they were making shit up?
No. This is just one failure to replicate. There would need to be many more attempts, an investigation, and actual proof of how they made shit up, to confirm they were making shit up.
Yup, you never know what the cause of the discrepancy could be. It seems even the original team could only get it to work 10% of the time anyway, and they were familiar with the process. Even with detailed instructions, another unfamiliar team may not be able to recreate it even that often.
Until they determine what factor is leading to the occasional creation of the product, it's effectively random whether they will create it or not. It could theoretically take 1000 tries to get it to work once. Or 1,000,000 times. But, it will probably take around 10.
That is, of course, if the product they claim to have made is real. If it isn't real, then they'll never get it. And, if they can figure out what exactly is making it or not, then they should be able to adapt the process to near perfect odds.
Reminds me of race conditions in programming.
1 in 15 times the bug happens and you can't figure it out, but if 2 asynchronous events happen to happen within 10ms of each other it breaks.
Could be some super specific timing on one of the steps where a discrepancy of a short time doesn't seem meaningful but is
Could be some super specific timing on one of the steps where a discrepancy of a short time doesn't seem meaningful but is
This is actually quite likely:
- The fabrication process consists of three separate heating-holding-cooling sequences, each one with a different holding temperature and timing.
- These sequences repeatedly anneal the material, changing its cristalline structure.
- The superconducting effect, is explained in terms of a slightly compressed crystalline structure creating a series of channels for electron tunnelling to happen.
If it wasn't all a scam, then it is very likely that a very precise temperature profile needs to be met in order for the effect to appear. It might've even been a fluke, and they themselves might be unable to fully reproduce it.
In the papers, they only got 4 samples, and still they seem to have messed around with some of them, so effectively no two samples had the same measurements (some were used for different tests, some were changed when used in the same test).
Well the good thing is they were able to reproduce it themselves so they won't drive themselves fully crazy trying to narrow that down, assuming not a scam.
I wonder how much you could automate that all to remove as much human factor as possible and be down to exact times if needed.
they were able to reproduce it themselves
It makes me wonder if they were, or how many attempts it took them to produce just 4 samples.
The process should be possible to automate completely, with the right temperature profile (commonly used in electronics, for reflow ovens and such).
But I wonder if there might be some other detail they might have forgotten to take into account... like maybe at some point you have to take the sample out with non-ferromagnetic tongs, but put it in with ferromagnetic ones, or flip it to the side, or align it with the heating coils of the oven, or whatever "irrelevant detail" like that.