Skip to content

Document sbt build issues with IDEs #653

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cloud-fan merged 4 commits into from
Dec 18, 2025
Merged

Document sbt build issues with IDEs #653

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b43face
Document sbt build issues with IDEs
gaogaotiantian Dec 12, 2025
42f0847
Clarify sbt usage in IDEs section
gaogaotiantian Dec 12, 2025
af8a5ad
Revise cache clearing instructions in developer-tools.md
gaogaotiantian Dec 15, 2025
3966da3
Update HTML file
gaogaotiantian Dec 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions developer-tools.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -494,6 +494,26 @@ in the Eclipse install directory. Increase the following setting as needed:
256M
```

<h4>IDE with sbt</h4>

When you use IDEs like VSCode or Cursor and you build from terminal with `sbt`,
Copy link
Member

@yaooqinn yaooqinn Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also provide tips for step-by-step setups for these IDEs? In the IDE setup section, we only have guidelines for IntelliJ. It's better to have instructions for them too

Copy link
Contributor Author

@gaogaotiantian gaogaotiantian Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest I don't have that information now. I'm definitely not opposed to that idea but I don't have enough material at hand and the main purpose of this PR is to document this specific behavior. We can of course have a better write up about how to set VSCode/Cursor for spark project.

a common issue you'll encounter is something like

```
java.lang.RuntimeException: Failed to find name hashes for
```

This is because multiple instances are trying to build the project simultaneously. Normally
the culprit is your Java/Scala plugin in your IDEs.

In order to fix this, first you need to disable auto-build of your plugins. Then you need to
clear all the artifacts with `sbt clean`. You'll also need to clear cache in

```
~/.m2
~/.ivy2/
```

<a name="nightly-builds"></a>
<h3>Nightly builds</h3>

Expand Down
18 changes: 18 additions & 0 deletions site/developer-tools.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,24 @@ <h4>Eclipse</h4>
256M
</code></pre></div></div>

<h4>IDE with sbt</h4>

<p>When you use IDEs like VSCode or Cursor and you build from terminal with <code class="language-plaintext highlighter-rouge">sbt</code>,
a common issue you&#8217;ll encounter is something like</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>java.lang.RuntimeException: Failed to find name hashes for
</code></pre></div></div>

<p>This is because multiple instances are trying to build the project simultaneously. Normally
the culprit is your Java/Scala plugin in your IDEs.</p>

<p>In order to fix this, first you need to disable auto-build of your plugins. Then you need to
clear all the artifacts with <code class="language-plaintext highlighter-rouge">sbt clean</code>. You&#8217;ll also need to clear cache in</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>~/.m2
~/.ivy2/
</code></pre></div></div>

<p><a name="nightly-builds"></a></p>
<h3>Nightly builds</h3>

Expand Down