Got a reviewer with an axe to grind that followed the paper from a previous conference submission. Hopefully they won’t be so egregious this time with reviews being public.
this post was submitted on 11 Nov 2023
1 points (100.0% liked)
Machine Learning
1 readers
1 users here now
Community Rules:
- Be nice. No offensive behavior, insults or attacks: we encourage a diverse community in which members feel safe and have a voice.
- Make your post clear and comprehensive: posts that lack insight or effort will be removed. (ex: questions which are easily googled)
- Beginner or career related questions go elsewhere. This community is focused in discussion of research and new projects that advance the state-of-the-art.
- Limit self-promotion. Comments and posts should be first and foremost about topics of interest to ML observers and practitioners. Limited self-promotion is tolerated, but the sub is not here as merely a source for free advertisement. Such posts will be removed at the discretion of the mods.
founded 1 year ago
MODERATORS
any advice for one reviewer bringing up totally unrelated papers and asking to explain the novelty of the method and the task itself when the other 3 reviewers understand the novelty from the jump?
got 3 3 3, does anyone know withdraw pros and cons ? Since everything is made public, wondering whether a withdraw without replying to the non-nonsensical reviewers would leave a bad impression to next conference submissions. Anyone has any advise?
take it as an opportunity to explain your work with extra pages (since there is not limit on the number of total rebuttal responses.) This is an unique opportunity for you to add more details, intuition, high-level overviews that you couldn't include in the main text. Moreover, it's open to public. So you get free extra pages for future submissions.
6555, any chance?
Anyone who get 7 reviews? I got 6, but one more review came out last night, so it becomes 6365566 with confidence 3133423.
What about 5, 5, 5, 8, 8? Negative reviewers > positive reviewers, while score above the borderline.
Got initial score of 855 but the reviewers just won't reply to the rebuttal... I even asked the AC and they sent a reminder for the reviewers but no one will reply :(
I think this reviewing system really has to change. Some papers get reviewers that are active and actually consider raising the score after looking at the rebuttal, and some get lazy reviewers that never look at the rebuttal and the score remains.
The reviewers know nothing will happen to them and they will again be selected as a reviewer next year, even if they do 'minimal work'. So frustrating :(
In the same boat, stuck with reviewers who don't want to engage. Don't know what to do. Does anyone with advice on how to get a reply from reviewers who are active?
There is basically no way. There are many reasons they might not reply, including being sick or occupied with their own borderline papers. Try to be professional about it: post a reminder and then let the Area Chairs (ACs) handle it. Most ACs are well aware of these types of situations. They will most likely place less weight on those non-engaging reviews. Good luck with your rebuttal!
That's great, healthy POV and advice. Thank you.
Although my thought process was a bit different, that is what I did as well.
Same. I also did many additional experiments, but complete silence from the reviewers. It really sucks and makes me feel a bit helpless. I'm used to reviewer roulette in general, but it feels even worse in this case because I see other submissions where all 4 reviewers are engaged and raised their score after the author response.
I would kind of understand disinterest by reviewers if my submission had very bad scores and would be hopeless anyway, but I got 8855 and was very positive that I could at least increase one of the 5s to a 6 with my response.
Same here..
I don't even know whether they decided to maintain the score after reading the rebuttal or they just forgot about it. At least there should be a checkbox to ensure that they have read the rebuttal.
6,3,8.
What are the chances?
That 3 is a pain in the butt. Best of luck!
With 8, 6, 5, 5, is it simply a 50/50 chance ? Just thinking how likely I’ll have to do a resubmission
Now since the discussion period is over, can the reviewers still reply back and change the scores?
If you look at the recent activity on openreview you can see that reviewers can still edit their reviews and change their scores. I don't think anyone can post completely new anonymous replies though.
Yes, one reviewer just won't respond :(
I sent the review responses after additional experiments. It took several days thus I could send it yesterday. I think the results are very good and reply the reviewers’ concerns perfectly. Will the score be improved even after response periods?
Anyone know what scores are considered borderline this year?
According to the following post, the borderline would be 5.75 - 6.0, I guess. Acceptance/rejection is not always decided by the score only, though.
- https://x.com/shaohua0116/status/1728158662265340047?s=20,
Did anyone's score change during the rebuttal period?
Yes, but just 1 out of 4 and just very slightly despite a very extensive rebuttal.
We got one reviewer to go from 5 -> 8.
Looks like the „Recent Activity“ tap was deactivated some hours ago.. Can there still be score changes?
How is spotlight/oral decided? From results in previous years, I found it unpredictable as some works with an average score of 6.25 can get spotlight while others with higher scores cannot.
Well, the reviewers are not exactly reliable, are they? For a good paper they are disinterested in or do not understand - they might just give 6s.
Does that mean AC will read the paper?
Depends on the AC.
I think a paper with 6.25 is actually close to the top 15%. These ratings are a bit weird.
First paper(co-first author) in my life as an undergrad:
8/6/6/6 (After rebuttal)