Skip to content

Sigils - IntelliLang Language Injection Support PoC #3671

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Sigils - IntelliLang Language Injection Support PoC #3671

wants to merge 1 commit into from

Conversation

polymorfiq
Copy link

@polymorfiq polymorfiq commented Dec 28, 2024 •
edited
Loading

First I wanted to say, great plugin! Thanks so much for maintaining it!

I do not know Java, Kotlin, IntelliJ Plugin development or really any of the associated tooling - so this is definitely just an example to potentially help development of features related to #695 and #2102

I have been recently using Phoenix LiveView extensively, and the lack of Language Injection and Syntax Highlighting for the ~H and ~L sigils makes that a little difficult

This PR seems to get Language Injection working for the two sigils, though I have not figured out how to get the HTML syntax highlighting working alongside it. Since IntelliJ knows it's HTML, I'm not entirely sure why it doesn't respect the syntax highlighting of that language 🤔

Screenshot 2024-12-27 at 11 16 11 PM

Notes:

Right now I have ~L pointed at EEx, though from my understanding it should be Heex or Leex

Ideally, ElixirSigilInjector is entirely un-needed or otherwise wouldn't specify languages (languageForSigil) - but without its existence, Intellij seemed to be using the markdown.Injector and thus ElixirSigilInjectionSupport was only being passed Heredoc elements. I could not get IntelliJ to ignore the org.elixir_lang.injection.markdown.Injector, so instead ElixirSigilInjectionSupport is currently overriding it as the default, for everything besides Heredocs.

The reason ElixirSigilInjector specifying languages should be unneeded, is that using the added sigilWithName pattern and elixirInjections.xml file, allows the user to override/attach arbitrary languages to arbitrary sigils, which is likely better than hardcoding which sigils should go to what language (like the ElixirSigilInjector seems to require):
Screenshot 2024-12-27 at 11 44 35 PM

Hope this helps! 🎉

@joshuataylor
Copy link
Collaborator

This is absolutely amazing, thank you for just diving in and getting dirty -- hope it wasn't too full of spiders!

Let me know if you need anything in future as well, I'd be happy to chat about any of this, especially as IntelliJ has been changing a lot of their Custom Language Support in the last year or so.

I'll review this ASAP. <3

@elepedus
Copy link

Can't wait to get this feature! IntelliJ has such great HTML support (hello, Emmet!) and losing all of that when working with sigils is really painful!

Please let me know if there's anything I can do to help get this over the line!

@joshuataylor
Copy link
Collaborator

joshuataylor commented Jan 11, 2025 via email

@joshuataylor
Copy link
Collaborator

Having a look at this, there seems to be an error with threading, looking into this.

Happens when I open a file (also love the sass, Jetbrains):

java.lang.IllegalStateException: Wow, you must not start injecting in one thread (null) and finish in the other (Thread[#149,ForkJoinPool.commonPool-worker-3,5,main])
  at com.intellij.psi.impl.source.tree.injected.InjectionRegistrarImpl.checkThreading(InjectionRegistrarImpl.java:156)
  at com.intellij.psi.impl.source.tree.injected.InjectionRegistrarImpl.addPlace(InjectionRegistrarImpl.java:126)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:110)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at org.elixir_lang.injection.ElixirSigilInjector.getLanguagesToInject(ElixirSigilInjector.kt:132)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.processInPlaceInjectorsFor(InjectedLanguageManagerImpl.java:498)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.probeElementsUpInner(InjectedLanguageUtilBase.java:237)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.lambda$probeElementsUp$0(InjectedLanguageUtilBase.java:217)
  at com.intellij.openapi.application.impl.ReadActionCacheImpl$allowInWriteAction$1.invoke(ReadActionCacheImpl.kt:41)
  at com.intellij.openapi.application.impl.ReadActionCacheImpl$allowInWriteAction$1.invoke(ReadActionCacheImpl.kt:41)
  at com.intellij.openapi.application.impl.ReadActionCacheImpl.allowInWriteAction(ReadActionCacheImpl.kt:29)
  at com.intellij.openapi.application.impl.ReadActionCacheImpl.allowInWriteAction(ReadActionCacheImpl.kt:41)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.probeElementsUp(InjectedLanguageUtilBase.java:216)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageUtilBase.enumerate(InjectedLanguageUtilBase.java:159)
  at com.intellij.psi.impl.source.tree.injected.InjectedLanguageManagerImpl.enumerateEx(InjectedLanguageManagerImpl.java:383)
  at com.intellij.codeInsight.daemon.impl.InjectedGeneralHighlightingPass.lambda$processInjectedPsiFiles$7(InjectedGeneralHighlightingPass.java:153)
  at com.intellij.concurrency.ApplierCompleter.processArrayItem(ApplierCompleter.java:126)
  at com.intellij.concurrency.ApplierCompleter.processArray(ApplierCompleter.java:207)
  at com.intellij.concurrency.ApplierCompleter.execAll(ApplierCompleter.java:176)
  at com.intellij.concurrency.JobLauncherImpl.lambda$invokeConcurrentlyUnderProgressAsync$3(JobLauncherImpl.java:108)
  at com.intellij.openapi.application.impl.AnyThreadWriteThreadingSupport.tryRunReadAction(AnyThreadWriteThreadingSupport.kt:333)
  at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:971)
  at com.intellij.concurrency.ApplierCompleter.lambda$wrapInReadActionAndIndicator$2(ApplierCompleter.java:158)
  at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$14(CoreProgressManager.java:674)
  at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:749)
  at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:705)
  at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:673)
  at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:79)
  at com.intellij.concurrency.ApplierCompleter.wrapInReadActionAndIndicator(ApplierCompleter.java:169)
  at com.intellij.concurrency.ApplierCompleter.lambda$wrapAndRun$1(ApplierCompleter.java:150)
  at com.intellij.openapi.application.impl.AnyThreadWriteThreadingSupport.executeByImpatientReader(AnyThreadWriteThreadingSupport.kt:544)
  at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:176)
  at com.intellij.concurrency.ApplierCompleter.wrapAndRun(ApplierCompleter.java:150)
  at com.intellij.concurrency.JobLauncherImpl.lambda$invokeConcurrentlyUnderProgressAsync$4(JobLauncherImpl.java:108)
  at com.intellij.concurrency.JobLauncherImpl.safeIterate(JobLauncherImpl.java:176)
  at com.intellij.concurrency.JobLauncherImpl.invokeConcurrentlyUnderProgressAsync(JobLauncherImpl.java:105)
  at com.intellij.concurrency.JobLauncherImpl.invokeConcurrentlyUnderProgress(JobLauncherImpl.java:53)
  at com.intellij.concurrency.JobLauncher.invokeConcurrentlyUnderProgress(JobLauncher.java:51)
  at com.intellij.codeInsight.daemon.impl.InjectedGeneralHighlightingPass.processInjectedPsiFiles(InjectedGeneralHighlightingPass.java:151)
  at com.intellij.codeInsight.daemon.impl.InjectedGeneralHighlightingPass.lambda$collectInformationWithProgress$4(InjectedGeneralHighlightingPass.java:84)
  at com.intellij.codeInsight.daemon.impl.ManagedHighlighterRecycler.runWithRecycler(ManagedHighlighterRecycler.java:85)
  at com.intellij.codeInsight.daemon.impl.InjectedGeneralHighlightingPass.collectInformationWithProgress(InjectedGeneralHighlightingPass.java:83)
  at com.intellij.codeInsight.daemon.impl.ProgressableTextEditorHighlightingPass.doCollectInformation(ProgressableTextEditorHighlightingPass.java:86)
  at com.intellij.codeHighlighting.TextEditorHighlightingPass.collectInformation(TextEditorHighlightingPass.java:67)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$2(PassExecutorService.java:427)
  at com.intellij.platform.diagnostic.telemetry.helpers.TraceKt.use(trace.kt:30)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$3(PassExecutorService.java:423)
  at com.intellij.openapi.application.impl.AnyThreadWriteThreadingSupport.tryRunReadAction(AnyThreadWriteThreadingSupport.kt:351)
  at com.intellij.openapi.application.impl.ApplicationImpl.tryRunReadAction(ApplicationImpl.java:971)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$doRun$4(PassExecutorService.java:413)
  at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$14(CoreProgressManager.java:674)
  at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:749)
  at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:705)
  at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:673)
  at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:79)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.doRun(PassExecutorService.java:412)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$0(PassExecutorService.java:388)
  at com.intellij.openapi.fileTypes.impl.FileTypeManagerImpl.cacheFileTypesInside(FileTypeManagerImpl.java:822)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.lambda$run$1(PassExecutorService.java:388)
  at com.intellij.openapi.application.impl.AnyThreadWriteThreadingSupport.executeByImpatientReader(AnyThreadWriteThreadingSupport.kt:544)
  at com.intellij.openapi.application.impl.ApplicationImpl.executeByImpatientReader(ApplicationImpl.java:176)
  at com.intellij.codeInsight.daemon.impl.PassExecutorService$ScheduledPass.run(PassExecutorService.java:386)
  at com.intellij.concurrency.JobLauncherImpl$VoidForkJoinTask$1.exec(JobLauncherImpl.java:266)
  at java.base/java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:507)
  at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
  at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1491)
  at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:2073)
  at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2035)
  at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)

@joshuataylor
Copy link
Collaborator

joshuataylor commented Jan 11, 2025 •
edited
Loading

Okay, so got the threading stuff sorted, though I can't figure out why it won't highlight!

It won't work inside of strings, it's really weird.

https://plugins.jetbrains.com/docs/intellij/language-injection.html#formatting

https://github.com/joshuataylor/intellij-elixir/tree/sigil-injected-languages-poke

edit:

Okay, so by default ELIXIR_SIGIL_UPPER_H overwrites the colour for injections.

/resources/colorSchemes/ElixirDefault.xml#L185-L189

    <option name="ELIXIR_SIGIL_UPPER_H">
        <value>
            <option name="FOREGROUND" value="99186C" />
        </value>
    </option>

Settings | Editor | Color Scheme | Elixir

image

Textual -> Sigil -> H

Disable "Foreground"

Injected language fragment can have its green colour removed as well:

image

The range needs to be fixed, but it's close.

@elepedus
Copy link

This is already looking so much better! Thank you! 🤩

@joshuataylor
Copy link
Collaborator

Managed to get it to work, what a pain! The documentation is really confusing and inconsistent.

For ~H, I set it to only match on the lines inside, as if you don't it'll match for example in the quotes in ~H """

SCR-20250114-bcae

HTML autocomplete works:

SCR-20250114-bipa SCR-20250114-bips

This lays the foundations for integrated autocomplete, using a mix of HTML+Elixir, e.g we could autocomplete:

def weather_greeting(assigns) do
  ~H"""
  <div title="My div" class={@class}>
    <p>Hello {@name}</p>
    <MyApp.Weather.city name="Kraków"/>
  </div>
  """
end

If you don't want the green background, you can turn that off, though note it turns it off for everything else as well that's injected.

SCR-20250114-bjcd-2 SCR-20250114-bjdu

Regex works as well:

image

--

I've also turned off the color schemes for ~H and ~r, they were blocking inspection. I believe that you can turn that off using InjectionBackgroundSuppressor, but I couldn't get it to work. 🤷

--

If you want to try my build and let me know, that'd be great - I've tested with the latest IntelliJ 2024.3.1.1 and it's been working well. I know RubyMine is really buggy with SDKs, hoping to get that fixed soon.

Elixir-20.0.1-pre+20250113164838.zip

@polymorfiq
Copy link
Author

@joshuataylor - Seems to work perfectly, for me!

Screenshot 2025-01-27 at 10 23 03 AM Screenshot 2025-01-27 at 10 22 24 AM Screenshot 2025-01-27 at 10 22 12 AM

It also worked for defining a custom sigil with a new language and so should work for folks that use DSLs or sigils specific to their codebase!:

Screenshot 2025-01-27 at 10 25 23 AM Screenshot 2025-01-27 at 10 26 39 AM

@polymorfiq
Copy link
Author

@joshuataylor - Feel free to commit your changes to this PR! I'm not precious about it or anything and would love to see the current state of it, for example if I can get InjectionBackgroundSuppressor working!

@cpoile
Copy link

cpoile commented Jan 31, 2025

@polymorfiq Thanks for your hard work on this! I'm assuming we would need to compile this branch to try it out?

@joshuataylor
Copy link
Collaborator

joshuataylor commented Jan 31, 2025 via email

@cpoile
Copy link

cpoile commented Jan 31, 2025

Ah, I missed the zip attached above 🤦‍♂️ Thanks, works much better than stock!

@joshuataylor joshuataylor mentioned this pull request Feb 1, 2025
Open
@joshuataylor
Copy link
Collaborator

Documented about adding the concept of Experimental Features, with the first being this.

I don't want to just turn it on by default right away, let's do a bit more testing and tweaking.

This should help soft-launch features and get feedback, and not be blocked as much.

--

Once I've tidied up the settings, I'll add my commits to this PR, thanks again for your hard work and patience @polymorfiq and everyone else!

@polymorfiq
Copy link
Author

polymorfiq commented Feb 2, 2025 •
edited
Loading

Sounds great @joshuataylor - What an awesome idea! Excited for the Experimental toggle!

@drl123
Copy link

drl123 commented Feb 11, 2025

Just a comment as I know this is WIP/experimental, but is there any way to get the HTML highlighting to work with function components (<.component />)? This is probably more of an IDE thing than the plugin itself. The sigil H stuff works pretty well with this zip other than the IDE doesn't consider tag names that start with a . as valid HTML tags or custom tags and so it does not decorate them. If I use a fully qualified component name (so it does not start with the dot) it highlights just fine.

Here <.banner> is a function component without a module name (does not highlight) and <CustomComponents.icon ...> is a function component where I passed the aliased module name (highlights). Oddly, it seems to be ok with the : for slots, just not the . for the component!

image

This is great progress though. One of the main reasons I was considering jumping to Zed or VS Code was because the rest of my team was inlining all of the templates. This is a HUGE improvement.

@cheezy
Copy link

cheezy commented Feb 13, 2025

+1 for the function component update.

@joshuataylor
Copy link
Collaborator

joshuataylor commented Feb 13, 2025 via email

@cheezy
Copy link

cheezy commented Feb 14, 2025

The only other item I see at this time is elixir code inside of {} has no highlighting. It would be nice if it has the same highlighting as elixir code inside functions.

@cheezy
Copy link

cheezy commented Mar 13, 2025

Documented about adding the concept of Experimental Features, with the first being this.

What does one have to do for the experimental features options to show up? I have the latest IntelliJ and the 20.0.1 plugin and I do not see the Elixir Experimental Settings option.

@bauffman
Copy link

Same here. The documentation doesn't say what to do to make the experimental features visible! @joshuataylor can you share that info or add it to the docs please?

@elepedus
Copy link

So it looks like this feature is currently only available from @joshuataylor's fork, by installing from zip.

It would be awesome to see this merged into main and get a new release cut so it's available directly through IntelliJ plugin updates! 🚀

@joshuataylor
Copy link
Collaborator

joshuataylor commented Mar 25, 2025 via email

@elepedus
Copy link

Hey, it's open-source, you don't owe us a thing! It will land when it suits you. I'm just excited to get my grubby mitts on this feature! :)

@drl123
Copy link

drl123 commented Apr 29, 2025 •
edited
Loading

@joshuataylor Any chance you could whip up a new zip that is compatible with the 2025 IDE's? I tried the zip above but it requires builds prior to 243.* and I'm on RM-251.23774.429. I'm running 20.0.1 and I don't see the experimental features options either. Hate to have to downgrade.

PS...appreciate all of the effort on this! ❤️

@joshuataylor
Copy link
Collaborator

I'll push a new release tonight (AU) with both this and the 2051.x support where it remove the untilBuild.

@cheezy
Copy link

cheezy commented May 2, 2025

I no longer see the plugin in the Marketplace. Running 2025.1 build 251.23774.435.

@joshuataylor
Copy link
Collaborator

I can't merge this as there are unsigned commits. I'll see what's going on

I've submitted the other build, which you can find in the other PRs, until this is merged.

@cheezy
Copy link

cheezy commented May 3, 2025

Is there a place where I can download a zip file of the latest plugin? It still is not showing up in the IntelliJ Plugins for me. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@polymorfiq @joshuataylor @elepedus @cpoile @drl123 @cheezy @bauffman