Guide to Reporting Software Errors

Reporting_Errors.md

Guide to Reporting Errors (CSCI 545, Spring 2024)

Note: this guide has been written for CSCI 545, Introduction to Robotics, taught by Daniel Seita in Spring 2024. We have a policy that students must state that they have read this guide before requesting software help. This is designed for the benefit of both the course staff but also for the students.

This is a short guide on how to report code-related errors. If you run into errors, following these guidelines will increase the chances that your issue gets quickly resolved. Otherwise, it might result in a protracted back-and-forth process where I continually ask for more information from you. As another bonus, following these steps may lead you to find the solution! There have been countless times when I was about to write a GitHub issue report or a StackOverflow question, but then I realized the solution due to following the below steps (especially the steps to reproduce).

Tip 1: Add More Information

Add more information than you think, and include steps to reproduce.

The single biggest problem that I have experienced is when students tell me something did not work, but they don't give me enough information about the error! They might just say "I ran into this error [...]" at a high level. I don't want that. What I need is the EXACT ERROR MESSAGE, copied and pasted (see the next tip).

Ideally, when you describe the error, you will also include information such as the machine you are using (and what operating system it runs) and then you will include the steps to reproduce, such as:

1. First clone this repository [...]
2. Then run this command to change directories: [...]
3. Then run this python script: [...]
4. Here is the traceback which shows the exact error: [...]

(Of course you will need to adjust this for your particular setting, but this is the idea.)

For examples of "good" error / issue reports, please see:

• Xingyu-Lin/softgym#9
• facebookresearch/theseus#566

Disclaimer: I am the author of those reports. But notice how the information that I provided in the above allowed the code maintainers to better understand and potentially reproduce my errors. For example, in the second link, I gave the complete steps to reproduce, and the maintainers immediately fixed a bug.

Once again: the EXACT ERROR MESSAGE.

Tip 2: Copy and Paste

Copy and paste, and avoid too many edits.

When you report an error, copy and paste what you see on the command line. Amazingly, I have had prior students who reported errors not by copying and pasting, but by literally trying to type (character by character) a long error traceback that they got, even if it was dozens of lines long! Again, please copy and paste.

I recommend copying and pasting text instead of using a screenshot. This is because with text, I can copy it easily and search it online myself or query GPT-4. With a screenshot, I can't "copy" the text from it as easily. Of course, screenshots can be very helpful if you are debugging something that's more visual (like a robot simulator). And yes, I am aware that we can probably ask GPT-4 to extract the text from the screenshot, but it's just another step that can be avoided with the raw text.

Furthermore, when copying and pasting, please be VERY CAREFUL about "fine-tuning" of the text to "clean" it because that could introduce more confusion and typos. I have had students report errors to me in which they first copied and pasted the raw output from the command line (good) but then they made some edits to the text, and in the process, they introduced typos that led to errors when I copied and pasted their text to reproduce the command (bad).

But what if you have so much information? You need to use your judgment about what to report. However, you can also use "pastebins" if needed which can better manage long pieces of text (see the next tip).

I also highly recommend screen recordings if you are sharing something that is very visual (e.g., a video). You can create a screen recording, upload the result to YouTube, and then share the YouTube link.

Tip 3: Post in a Clean Way

Note: some of this might not be supported on Piazza, so please adjust as needed.

Post in a "clean" way with backticks, toggles, etc.

When reporting error messages, please use the backticks. That means your text will look like this if it does not create a new line, or

like this

if you choose to make a new line. This is cleaner than copying an error because each character is the same width and it ignores effects that might alter the text such as (if using Markdown) asterisks * resulting in italic text, etc. For example compare this:

(softgym) seita@triton1:~/softgym (master) $ python examples/random_env.py --env_name ClothFlatten
Traceback (most recent call last):
  File "examples/random_env.py", line 5, in <module>
    from softgym.registered_env import env_arg_dict, SOFTGYM_ENVS
  File "/home/seita/softgym/softgym/registered_env.py", line 1, in <module>
    from softgym.envs.pour_water import PourWaterPosControlEnv
  File "/home/seita/softgym/softgym/envs/pour_water.py", line 4, in <module>
    import pyflex
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /home/seita/softgym/PyFlex/bindings/build/pyflex.cpython-36m-x86_64-linux-gnu.so)
(softgym) seita@triton1:~/softgym (master) $ ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu11.2) 2.23
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
(softgym) seita@triton1:~/softgym (master) $ 

versus this, which is the same text as the above, except copied and pasted below without the backticks:

(softgym) seita@triton1:/softgym (master) $ python examples/random_env.py --env_name ClothFlatten Traceback (most recent call last): File "examples/random_env.py", line 5, in from softgym.registered_env import env_arg_dict, SOFTGYM_ENVS File "/home/seita/softgym/softgym/registered_env.py", line 1, in from softgym.envs.pour_water import PourWaterPosControlEnv File "/home/seita/softgym/softgym/envs/pour_water.py", line 4, in import pyflex ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /home/seita/softgym/PyFlex/bindings/build/pyflex.cpython-36m-x86_64-linux-gnu.so) (softgym) seita@triton1:/softgym (master) $ ldd --version ldd (Ubuntu GLIBC 2.23-0ubuntu11.2) 2.23 Copyright (C) 2016 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Written by Roland McGrath and Ulrich Drepper. (softgym) seita@triton1:~/softgym (master) $

Which one is easier to read? Two quick things to note:

• You can do this both in Notion and in Markdown. Please follow the same principle for both.
• Notice how the above error also includes the working directory information. It also informs the reader that I am in the softgym conda environment (due to the (softgym) text), it tells me which command I ran, it tells me the error, etc. All this information can be helpful!

Another resource you might want to be aware of are toggles. You can do this to "hide" long blocks of text.

Like this. Click on the arrow.

(softgym) seita@triton1:~/softgym (master) $ python examples/random_env.py --env_name ClothFlatten
Traceback (most recent call last):
  File "examples/random_env.py", line 5, in <module>
    from softgym.registered_env import env_arg_dict, SOFTGYM_ENVS
  File "/home/seita/softgym/softgym/registered_env.py", line 1, in <module>
    from softgym.envs.pour_water import PourWaterPosControlEnv
  File "/home/seita/softgym/softgym/envs/pour_water.py", line 4, in <module>
    import pyflex
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /home/seita/softgym/PyFlex/bindings/build/pyflex.cpython-36m-x86_64-linux-gnu.so)
(softgym) seita@triton1:~/softgym (master) $ ldd --version
ldd (Ubuntu GLIBC 2.23-0ubuntu11.2) 2.23
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.
(softgym) seita@triton1:~/softgym (master) $ 

What if it is still SUPER long? I recommend Pastebin: https://pastebin.com/.

Tip 4: Cite References

Cite which resources and websites you have consulted.

If you run into an error, use your favorite search engine or AI assistant to help. These days, ChatGPT is really helpful with a lot of common software errors such as "package not found." When you report your error, it helps if you add links to references which have the solution. It might be as simple as needing someone with sudo access to install a package for you on a machine.