nl2code / coder Goto Github PK

Congratulations on the great results on swebench! We would like to reproduce this work. Do you have any plans to release the code and data (instance id of the small set for the ablation study)?

Wow, congratulations on the great results on SWE-Bench Lite! We would love to reproduce the results, are there plans to release the code?

Please share your source code

Hi, thanks for the nice work! Do you have a plan to release the source code of CodeR, so that people can learn more about it?

Congratulations on the release and thank you for citing the AutoCodeRover paper!

The paper was a great read. After going through it, I have a couple of clarification questions:

1. The Plan D part above Figure 3 mentions that "Plan D takes a test-driven approach with a ground truth test for issues (such as 'fail-to-pass' and 'pass-to-pass' tests in SWE-bench)."
   Does this mean the developer-written tests for the issue (i.e. the test_patch field in SWE-bench instances) were provided to CodeR?

2. Section 2 mentions that "Action 18 retrieves the top-1 similar issue and its corresponding patch by description." (action 18 is related issue retrieval from Table 1).
   This is an interesting approach! I'm curious how you defined "similarity" between issues - was this using a RAG-based approach on the issue descriptions? Besides, how did you construct the corpus of issues to retrieve from?

Thank you very much in advance for your time and assistance!

Is this not clear from paper
Are you going to publish it?

Hi, does the plan D mentioned in your paper use "fail-to-pass" tests that are actually used to evaluate the patches?

If so, this would be kind of unfair because most of the other methods do not use those.

Could you maybe specify how many instances in the paper are solved by Plan D?