firefox: add support for declarative extension configurations #6389

HPsaucii · 2025-01-31T12:13:32Z

Description

Adds support for declarative extension configuration through JSON by disabling IDB (check commit ca31bdd for more details)

Checklist

Change is backwards compatible.
Code formatted with ./format.
Code tested through nix-shell --pure tests -A run.all
or nix build --reference-lock-file flake.lock ./tests#test-all using Flakes.
Test cases updated/added. See example.
Commit messages are formatted like
```
{component}: {description}

{long description}
```
See CONTRIBUTING for more information and recent commit messages for examples.
If this PR adds a new module
- Added myself as module maintainer. See example.

Maintainer CC

@rycee @brckd

This commit refactors programs.firefox.profiles.<name>.extensions in order to support both installation of extensions (addons) and their configuration. It does this by setting the `extensions.webextensions.ExtensionStorageIDB.enabled` user_pref to false. When this preference is set to false, support for storing extension settings in sqlite databases, also known as IndexedDB or IDB, is reverted back to the JSON format present in firefox versions prior to version 63, as seen here: https://blog.mozilla.org/addons/2018/08/03/new-backend-for-storage-local-api/ IndexedDB was made the default due to performance improvements, but had the consequence of removing any possibility of declarative extension configuration without the assistance of firefox's policy system. The policy system is supported by a small amount of extensions, such as uBlock Origin, but has to be explicitly supported. Even when supported, it provides significantly less granular control when compared to the JSON storage format.

HPsaucii · 2025-01-31T12:14:59Z

Not backwards compatible due to previous lack of forwards compatibility previously in programs.firefox.profiles..extensions.

Added myself as module maintainer, but not a new module

HPsaucii · 2025-01-31T12:15:39Z

closes #4618

HPsaucii · 2025-01-31T12:20:19Z

(whoops, tests failed because i forgot to commit the description :p, fixed now)

brckd · 2025-01-31T14:52:16Z

Thank you for the promising pull request! I didn't even expect #4618 to ever be resolved at this point, but now I'm looking forward to it!

Because this change is not trivial I'll just summarize what I've gathered so far:

This adds the profiles.<profile>.extensions.settings.<extension> option which writes its contents to browser-extension-data/<extension>/storage.js
Therefore it moves the previous profiles.<profile>.extensions option to profiles.<profile>.extensions.packages
In order for the browser to use the storage.js file, the extensions.webextensions.ExtensionStorageIDB.enabled policy is set to false. This might introduce some regression in runtime-performance for extensions.

Overall it looks like a good approach, but I have some minor concerns.

Ideally we would like to keep some backwards compatibility so users have some time to transition. It might be possible for both the old and the new options layout to coexist using lib.types.either.
Have you tested this with any extension? Also consider adding unit tests somewhere in tests/modules/programs/firefox/profiles that test whether the correct storage.js is generated.
Does this in theory work for all extensions, like those using the policy system? That would be important for consistency.
Do you know whether the JSON format be converted to the IndexedDB one? This way we could avoid the performance regression. That shouldn't be a huge concern though.

ambroisie · 2025-01-31T15:18:58Z

Ideally we would like to keep some backwards compatibility so users have some time to transition. It might be possible for both the old and the new options layout to coexist using lib.types.either.

BTW: that's what lib.types.coercedTo is for.

brckd · 2025-01-31T15:55:46Z

Ideally we would like to keep some backwards compatibility so users have some time to transition. It might be possible for both the old and the new options layout to coexist using lib.types.either.

BTW: that's what lib.types.coercedTo is for.

Yea this looks perfect! It would be nice if this allowed to raise warnings in the event the config was coerced though.

HPsaucii · 2025-01-31T16:08:12Z

Hiya @brckd thanks for the response, I've also been really excited about the potential for having this implemented.

Your summary is mostly correct, only nitpick is that extensions.webextensions.ExtensionStorageIDB.enabled is a user preference, as in from about:config and modified using the user.js file, and not what would typically be called a policy.

Keeping backwards compatibility would be a plus, even if its only temporary. I did not know that the libs you and @ambroisie suggested were a thing, and i'll get around to implementing it later today (GMT+0)

I've tested this pretty thoroughly, including with the following extensions:

uBlock Origin
uMatrix
Redirector
Stylus
Decentraleyes

But it should work with any extension. This is because the extensions.webextensions.ExtensionStorageIDB.enabled preference changes how firefox and its derivatives handle extension requests to the storage API, but doesn't change the API itself.
I will still implement some sanity checks in the form of tests, again, later today.

Extensions that utilise the policy system (which is barely any, with uBlock Origin being the gold in the rough) will have their own unique behaviour with both policies and declarative changes to storage.js enabled simultaneously, and due to no upstream cohesive solution from mozilla, will vary from implementation to implementation. Atleast with uBlock Origin, it appears to prioritise policies over storage, regardless of whether its using storage.js or IDB. Regardless though, there shouldn't be any issues.

As far as I'm aware, it is not possible to manually or declaratively modify the IndexedDB storage. This is, to the best of my memory, because the storage itself has to be signed by mozilla, and cannot be mitigated.

Some extra things I want to clear up:

The performance difference should be minimal, at most a handful of milliseconds, and would only be present if the user set any declarative extension settings in the first place.

Additionally, a similar implementation might be possible for chromium based browsers. I have no proof of this and it's just purely speculation, but I hope to investigate it further eventually.

brckd · 2025-01-31T19:17:10Z

Thanks for clarifying!

i'll get around to implementing it later today (GMT+0)

Take as much time as you need, I'll be asleep and on vacation anyways ;)

I've tested this pretty thoroughly, including with the following extensions:

uBlock Origin

uMatrix

Redirector

Stylus

Decentraleyes

This is already a good selection of extensions one would configure declaratively! I would personally also be interested in the Containers, Dark Reader and Libredirect extensions and will test these once we get this PR done.

I will still implement some sanity checks in the form of tests, again, later today.

Thanks, but again, no pressure.

Extensions that utilise the policy system (which is barely any, with uBlock Origin being the gold in the rough) will have their own unique behaviour with both policies and declarative changes to storage.js enabled simultaneously, and due to no upstream cohesive solution from mozilla, will vary from implementation to implementation. Atleast with uBlock Origin, it appears to prioritise policies over storage, regardless of whether its using storage.js or IDB. Regardless though, there shouldn't be any issues.

Alright, so this would indeed work consistently across extensions and keeps compatability with the policy-based approach. Perfect!

As far as I'm aware, it is not possible to manually or declaratively modify the IndexedDB storage. This is, to the best of my memory, because the storage itself has to be signed by mozilla, and cannot be mitigated.

The performance difference should be minimal, at most a handful of milliseconds, and would only be present if the user set any declarative extension settings in the first place.

In that case the JSON approach is absolutely sufficient.

Additionally, a similar implementation might be possible for chromium based browsers. I have no proof of this and it's just purely speculation, but I hope to investigate it further eventually.

In my experience, Chromium is a bit less customizable e.g. when it comes to theming, but maybe you can surprise me once again!

HPsaucii · 2025-01-31T20:01:17Z

@brckd i've implemented your suggestions, lmk if there's anything additional i should do 👍
(yes, i know, forgot to add a description for the second time now 🤦)

brckd

Just some minor suggestions. I also see a lot of formatting changes. Did you run nixfmt?

modules/programs/firefox/mkFirefoxModule.nix

HPsaucii · 2025-02-01T11:13:49Z

@brckd yeah i did, i think the issue might be that nixfmt doesn't play super nicely with git diff when moving stuff around, the test should have failed if it wasn't formatted according to ./format -c, just going to review your changes rq (yes, i was tired when i wrote all this, can't you tell :p)

Co-authored-by: bricked <[email protected]>

HPsaucii · 2025-02-01T15:07:38Z

woops, wrong button, about to commit

brckd

Just this one little fix and the PR should be good to go!

modules/programs/firefox/mkFirefoxModule.nix

Co-authored-by: bricked <[email protected]>

brckd

Everything is looking good now. @rycee I think you can merge this whenever you're ready.

kira-bruneau · 2025-02-04T19:26:51Z

modules/programs/firefox/mkFirefoxModule.nix

+              packages = mkIf (builtins.length packages > 0) (warn ''
+                In order to support declarative extension configuration,
+                extension installation has been moved from
+                programs.firefox.profiles.<profile>.extensions


Just a small nit - any instances of programs.firefox in the documentation should be replaced with ${moduleName}, since this file meant to be re-usable in firefox forks.

HPsaucii added 2 commits January 31, 2025 09:04

maintainers: add HPsaucii

f6ab771

firefox: fix failing tests due to lack of description

2be07e8

HPsaucii changed the title ~~Firefox extensions~~ firefox: add support for declarative extension configurations Jan 31, 2025

HPsaucii added 2 commits January 31, 2025 19:47

firefox: implement tests, improve backwards compat

4f0e9fc

firefox: fix failing tests due to lack of description

678dd3d

brckd suggested changes Feb 1, 2025

View reviewed changes

HPsaucii and others added 4 commits February 1, 2025 11:17

firefox: correct documentation for extension config

78dc08e

Co-authored-by: bricked <[email protected]>

firefox: improved description for extension config

b964ac4

Co-authored-by: bricked <[email protected]>

firefox: fix extension config test

38e7fe3

Co-authored-by: bricked <[email protected]>

firefox: correct documentation for extensions

ce25cf4

Co-authored-by: bricked <[email protected]>

HPsaucii requested a review from brckd February 1, 2025 15:07

HPsaucii added 2 commits February 1, 2025 15:12

firefox: move config examples to dedicated section

3243875

firefox: make extension config override optional

06b77de

brckd suggested changes Feb 1, 2025

View reviewed changes

modules/programs/firefox/mkFirefoxModule.nix Outdated Show resolved Hide resolved

Firefox: make extension example literal

753afe9

Co-authored-by: bricked <[email protected]>

HPsaucii requested a review from brckd February 2, 2025 06:53

brckd approved these changes Feb 2, 2025

View reviewed changes

brckd mentioned this pull request Feb 4, 2025

firefox: split bookmarks into separate submodule file #6402

Open

6 tasks

kira-bruneau reviewed Feb 4, 2025

View reviewed changes

teto added the program: firefox label Feb 4, 2025

khaneliman added the merge conflict label Feb 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

firefox: add support for declarative extension configurations #6389

firefox: add support for declarative extension configurations #6389

HPsaucii commented Jan 31, 2025 •

edited

Loading

HPsaucii commented Jan 31, 2025

HPsaucii commented Jan 31, 2025

HPsaucii commented Jan 31, 2025 •

edited

Loading

brckd commented Jan 31, 2025 •

edited

Loading

ambroisie commented Jan 31, 2025

brckd commented Jan 31, 2025

HPsaucii commented Jan 31, 2025 •

edited

Loading

brckd commented Jan 31, 2025

HPsaucii commented Jan 31, 2025

brckd left a comment

HPsaucii commented Feb 1, 2025

HPsaucii commented Feb 1, 2025

brckd left a comment

brckd left a comment

kira-bruneau Feb 4, 2025 •

edited

Loading

firefox: add support for declarative extension configurations #6389

Are you sure you want to change the base?

firefox: add support for declarative extension configurations #6389

Conversation

HPsaucii commented Jan 31, 2025 • edited Loading

Description

Checklist

Maintainer CC

HPsaucii commented Jan 31, 2025

HPsaucii commented Jan 31, 2025

HPsaucii commented Jan 31, 2025 • edited Loading

brckd commented Jan 31, 2025 • edited Loading

ambroisie commented Jan 31, 2025

brckd commented Jan 31, 2025

HPsaucii commented Jan 31, 2025 • edited Loading

brckd commented Jan 31, 2025

HPsaucii commented Jan 31, 2025

brckd left a comment

Choose a reason for hiding this comment

HPsaucii commented Feb 1, 2025

HPsaucii commented Feb 1, 2025

brckd left a comment

Choose a reason for hiding this comment

brckd left a comment

Choose a reason for hiding this comment

kira-bruneau Feb 4, 2025 • edited Loading

Choose a reason for hiding this comment

HPsaucii commented Jan 31, 2025 •

edited

Loading

HPsaucii commented Jan 31, 2025 •

edited

Loading

brckd commented Jan 31, 2025 •

edited

Loading

HPsaucii commented Jan 31, 2025 •

edited

Loading

kira-bruneau Feb 4, 2025 •

edited

Loading