Proposed conventions for IDA configurations

Plugin-Configuration-Standards.md

The Problem

When writing and using IDA plugins, configurations tend to be quite a mess. With each plugin having it's own:

1. Color scheme
2. Hotkeys
3. Configuration file format
4. Configuration location

(And that's when you have a seprtate configuration, and not some variables in the plugin itself).

In the current situation, each developer works as he sees fit, and the users usually end up with the unmodified defaults, as well as a large number of .some-plugin-confuguration files.

The Solution

To solve this, we need to create standards for plugin configuration, as well as C and Python libraries that follow them.

Key Features

1. Configurations should be easily exportable and modifiable (a YAML file might be a good solution)
2. Should allow user/project based heirarcy:
3. Global configuration, stored in the IDA directory
4. User configuration, in the user directory
5. Per-directory config, stored in the same directory as the .idb itself
6. Per-IDB config, stored as net-nodes in the IDB itself.
7. Should allow global-defaults, and per-plugin modifications (like default color schemes for highlighting).

C&C Welcome

If you feel this is relevant to you, please comment so that we can improve on those ideas before going ahead an implementing them.

Ok, #2 makes sense to me. Sounds like keeping all the settings under one organization/application QSettings instance with multiple subgroups should work. Export-by-plugin will be an API, with the .INI file format as the intermediate representation.
#4: Although the C++ QSettings class supports multiple types for value data, the Python layer transparently encodes/decodes QVariants. More importantly, the netnode API is picky about datatypes: you can only store bytes objects as values. So, we'll have to do some encoding/decoding anyways. Therefore, I propose we pick a format that always encodes to string/bytes, and we'll have the same behavior with every backend.

So, #4. I see no advantage to using msgpack, "fast-and-small" is not significant when talking about small and mostly-static data. So I think JSON would be preferable, adding no dependencies to the project.

I've updated the module to address issues #1-4.

You must now instantiate IDASettings instances with a single parameter that is the name of the plugin whose settings you'll fetch. All settings are placed in the same QSettings instance, under separate groups. This reduces the number of registry keys and config files created.

You can now fetch the list of names of plugins that have settings using the class properties IDASettings.system_plugin_names, IDASettings.user_plugin_names, etc. I don't think this is the most beautifully-named API, but it works. It allows you to enumerate all plugin settings instances, and from there, you can enumerate the settings key-value pairs themselves.

Values are now transparently JSON encoded and decoded as they are stored and fetched from the backends. This allows for values of types like: int, float, list, dict, etc. The netnode backend does not support values larger than 1024 bytes, so users are encouraged to store small values rather than large complex items.

I recommend bringing up additional issues specific to the module as github issues, so we both receive notifications. This gist remains a great place to discuss the general concept, however.

Regarding the 1024 bytes limit. I find it a bit problematic. The first plugin I wanted to use ida-settings is my plugin-loader. The loader currently holds a list of paths for all the plugins it loads. Paths are naturally long, so the data is easily over 1024 bytes. Any solution that resolves that on the plugin side is bound to be messy.

Any thoughts on that? I am leaning towards letting a JSON dump spread over several nodes.

Yes, I think this is a pretty annoying limitation.

Spreading the JSON over multiple netnodes is the same solution I came to a while back for another project. I have an existing library for netnode access that supports arbitrarily sized values, and handles the allocation of nodes. It would be a good fit for this project, and should solve the issue.

I'll spawn off another project to hold this code. You've provided some great feedback on ida-settings; would you mind giving the new project a review as well? I've enjoyed having a second pair of eyes note issues, suggest changes, and provide logical arguments.