Is there a charting library where I can get the mouse coordinates in plot units?

[dmenne]

First Check

• I added a very descriptive title here.
• I have checked the FAQs for a similar question and didn't find it.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the NiceGUI documentation, with the integrated search.
• I already searched online "How to X in NiceGUI" and didn't find any information.

Example Code

def on_click(event):
        if event.xdata is not None and event.ydata is not None:
            logger.info(f"Clicked at: ({event.xdata:.2f}, {event.ydata:.2f})")

Description

I am coming from R, where even the most basic charts can return the mouse position in the chart in user coordinates (i.e. not simple in pixels).

Trying this with one of the charting libraries in NiceGUI/python.

• Matplotlib/Pyplot: Pixels simple, user coordinates look difficult, see How to get mousedown position in ui.matplotlib #3063
• Plotly: I know from the R library, that it is easy to find the user values for a point, but what for some arbitrary position in space? There are some crazy methods around with and random invisible pixel color canvas
• Echart: There is an "on_click_point", see plotly, but how to get the user coordinates of an arbitrary background pixel

Any suggestion for a library to do this easily? Something like:

NiceGUI Version

Any

Python Version

3.13.9

Browser

Chrome

Operating System

Windows

Additional Context

No response

[dmenne]

Looks like Transform.inverted will do the trick:

Added later: Sorry for not reading

#3063 (reply in thread)

carefully, which had used this method before.

[dmenne]

Nice to know. Currently, I only need x, which is easy. Please post here when there is more. I do not close it, as a reminder to both of us.

Thanks for working on that great toolkit. I am in R since 28 years, these are my first steps into the python world where sometimes simple things are complicated.

[dmenne]

I am wondering why it's not easier to use

https://matplotlib.org/stable/gallery/event_handling/coords_demo.html

[dmenne]

I have to retract my "x, which is easy". The suggestion in #3063 looks like it is correct, but it's only approximate for HTML plots and can be deceivingly correct in the middle of the plot. I also tried clientX, layerX, and the others.
In the picture, I always clicked on the red lines.

The resulting x shown as vertical grey lines is too high for 90, too low for 120, and correct for 100, so there is some scaling issue. And it depends on the scaling when the image rescale in a narrow browser window.

Still searching for a solution, even if it is a workaround for the simple plot. This was meant for an interactive teaching tool, where students had to estimate medians, means, quartiles. I never thought getting the mouse position in a plot would be such a nightmare, after having done this a million times before.

[falkoschindler]

@dmenne I totally understand your frustration. Getting precise coordinates in the data frame shouldn't be so hard.
Maybe ui.interactive_image can help in simple cases. But a proper solution for a charting library would be great!

[dmenne]

Thanks, I had seen this, and forgotten that it could be used. Will try.

[dmenne]

In case someone needs it (the scaling is straightforward, but tedious to get right):

from PIL import Image  # noqa
from nicegui import ui, events
import matplotlib.pyplot as plt

format = "svg"
format = "png"
img_file = f"mousecoords.{format}"


# For simplicity, we only care for the x-value
left = 0.1
right = 0.9
top = 0.95
bottom = 0.25
width = 1000
height = 200
x_min = 80
x_max = 120

px = 1 / (plt.rcParams["figure.dpi"])
fig, ax = plt.subplots(figsize=(width * px, height * px))
fig.subplots_adjust(bottom=bottom, top=top, left=left, right=right)
ax.set_xlabel("Systolic blood pressure (mmHg)")
ax.set_xlim(x_min, x_max)
ax.set_ylim(-1, 1)
ax.axvline(100)
ax.axhline(0, color="grey", linewidth=0.2)
plt.savefig(img_file)  # Use file detour for better testing

# Better hide this in a closure....
def x_scaled(x, x_min=x_min, x_max=x_max, left=left, right=right, img_width=width):
    return (x_max - x_min) * (x - left * img_width) / (
        (right - left) * img_width
    ) + x_min


def mouse_handler(e: events.MouseEventArguments):
    ui.notify(f"image: {e.image_x:.1f}, user: {round(x_scaled(e.image_x),1))} ")


image = ui.interactive_image(
    source=img_file,
    on_mouse=mouse_handler,
    events=["mousedown"],
)

ui.run()

[dmenne]

Too bad, the scaling is correct for the png version, but not for the svg version. Click on the right border to see the problem.

I assume it has to do with the way the ticks are taken into account. I will use the png version for now.

Suggestions for correction are highly welcome, I would prefer the svg version.

[falkoschindler]

@dmenne I found the following shorter solution (at least for PNG-based rendering):

import io
import matplotlib.pyplot as plt
from PIL import Image
from nicegui import events, ui

fig, ax = plt.subplots()
fig.set_dpi(100)
ax.plot([80, 100, 120], [1, 2, 3], 'o-')

buffer = io.BytesIO()
plt.savefig(buffer)
buffer.seek(0)
img = Image.open(buffer)

def handle_click(e: events.MouseEventArguments):
    x_data, y_data = ax.transData.inverted().transform((e.image_x, img.height - e.image_y))
    ui.notify(f'image: ({e.image_x:.1f}, {e.image_y:.1f}), data: ({x_data:.1f}, {y_data:.1f})')

ui.interactive_image(img, on_mouse=handle_click)

ui.run()

[falkoschindler]

Here is an alternative for SVG:

import base64
import io
import matplotlib.pyplot as plt
from nicegui import events, ui

fig, ax = plt.subplots()
fig.set_dpi(96)
ax.plot([80, 100, 120], [1, 2, 3], 'o-')

buffer = io.BytesIO()
plt.savefig(buffer, format='svg')
svg_b64 = base64.b64encode(buffer.getvalue()).decode()
img_height = fig.get_figheight() * fig.get_dpi()

def handle_click(e: events.MouseEventArguments):
    x_data, y_data = ax.transData.inverted().transform((e.image_x, img_height - e.image_y))
    ui.notify(f'data: ({x_data:.1f}, {y_data:.1f})')

ui.interactive_image(f'data:image/svg+xml;base64,{svg_b64}', on_mouse=handle_click)

ui.run()

[dmenne]

Peinlich... I had tried the inverted transform, and had failed (but not with the interactive one). Both great. Please keep in docs.

[dmenne]

See also https://forum.posit.co/t/how-to-use-event-register-for-plotly-mouse-click-events/208973/5?u=dmenne

[dmenne]

Just curious: Why is the magical fig.set_dpi(96) required to get the offset correct? When I removed or replaced it, and readout was incorrect.

[falkoschindler]

Good question. If I remember correctly, it has to do with SVG's default DPI of 72 and the ratio of points to pixels of 1.333. Or put differently: 1 inch = 96 pixels.