Using on Mac OS X

[guibor]

Hi!

When I use this on Mac OS, I get the following error with mbsync

Error performing SASL authentication step: SASL(-1): generic failure: Unable to find a callback: 18948

I know this is probably an issue with mbsync, but perhaps there's a way to avoid it without changing isync code?

Thanks!

[pejaab]

Hi,
I expect this has to do with OS X SASL setup. I am currently trying to figure it out myself. As far as I understand one needs to install https://www.cyrusimap.org/sasl/ (can be done via brew). This is where I am stuck currently, as using pluginviewer the only plugin I am shown is EXTERNAL. I think ideally, one would see all plugins coming with cyrus sasl. My assumption is, if that works, one would need to install this xoauth2 plugin into a place where cyrus sasl finds it as well. Then one should be good to go.
As pointed out, I am not there myself. So the description is only based on assumption and what I have read so far ( mostly: https://www.reddit.com/r/commandline/comments/muhnqb/request_for_help_with_mbsync_and_xoauth2_for/ and the documentation of both cyrus sasl and xoauth2 itself).
Maybe it points you in the correct direction... Any help would be highly appreciated

[guibor]

I have reached more or less the same state where I am shown EXTERNAL and now other plugins.

That's as far as I got.

[leothelocust]

Did anyone discover a solution for this? @guibor @pejaab ?

[guibor]

I haven't. Perhaps @moriyoshi might have an idea?

[moriyoshi]

Sorry for leaving this behind too long. The answer depends on the setup, for example where the accompanying SASL library comes from, either the system provided or the one installed on your own, through homebrew, MacPorts, or by building it yourself. Anyways, the number appeared in the error message Unable to find a callback: 18948 indicates the constant number called "callback ID" that is designated by the header file as SASL_CB_* and instructs the SASL library what operation it should delegate to the underlying implementation. 18948 corresponds to 0x4a04, and what drew my attention the most is there is no such designation in the original Cyrus-SASL, but there is, in the Apple's implementation where you can find SASL_CB_OAUTH2_BEARER_TOKEN for that value.

So all I can assume at the moment is that you are most likely using the system SASL library and I need to know more context about why this callback ID is propagated.

[guibor]

Thanks @moriyoshi - what about the context would be helpful? It does seem that the system library is called but I am unsure how to make mbsync call the right library

[ldavison]

Did you compile mbsync --with-sasl=/path/to/cyrus? It compiles and no longer calls the Apple provided implementation, but I wasn't able to get it to work. Running pluginviewer only shows EXTERNAL.

It wasn't immediately clear to me why this was the only plugin listed, so I ended up patching mbsync. By adding the above mentioned callback, I was able to get mbsync working with the Apple provided implementation. Certainly not ideal, but at least it works until I can figure something else out.

[guibor]

Hi @ldavison this sounds promising, do you have more details on the patch? What exactly did you change in mbsync's source?

[guibor]

Specifically, the area of code is in drv_imap.c and seems to be

static sasl_callback_t sasl_callbacks[] = {
	{ SASL_CB_USER,     NULL, NULL },
	{ SASL_CB_AUTHNAME, NULL, NULL },
	{ SASL_CB_PASS,     NULL, NULL },
	{ SASL_CB_LIST_END, NULL, NULL }
};

static int
process_sasl_interact( sasl_interact_t *interact, imap_server_conf_t *srvc )
{
	const char *val;

	for (;; ++interact) {
		switch (interact->id) {
		case SASL_CB_LIST_END:
			return 0;
		case SASL_CB_USER:  // aka authorization id - who to act as
		case SASL_CB_AUTHNAME:  // who is really logging in
			val = ensure_user( srvc );
			break;
		case SASL_CB_PASS:
			val = ensure_password( srvc );
			break;
		default:
			error( "Error: Unknown SASL interaction ID\n" );
			return -1;
		}
		if (!val)
			return -1;
		interact->result = val;
		interact->len = strlen( val );
	}
}

what changes did you make to this area before compiling --with-sasl=/path/to/cyrus?

Do you mean adding a case to the function as in

   case SASL_CB_OAUTH2_BEARER_TOKEN:
	return  0x4a04;

or something else?

Thanks @ldavison!

[ldavison]

@guibor - The following diff will apply against the mbsync tag for v1.4.4, which will get mbsync working with Apple's implementation:

diff --git a/src/drv_imap.c b/src/drv_imap.c
index c5a7aed..b8ac65a 100644
--- a/src/drv_imap.c
+++ b/src/drv_imap.c
@@ -2195,6 +2195,9 @@ static sasl_callback_t sasl_callbacks[] = {
        { SASL_CB_USER,     NULL, NULL },
        { SASL_CB_AUTHNAME, NULL, NULL },
        { SASL_CB_PASS,     NULL, NULL },
+#ifdef __APPLE__
+       { SASL_CB_OAUTH2_BEARER_TOKEN, NULL, NULL },
+#endif
        { SASL_CB_LIST_END, NULL, NULL }
 };

@@ -2212,6 +2215,9 @@ process_sasl_interact( sasl_interact_t *interact, imap_server_conf_t *srvc )
                        val = ensure_user( srvc );
                        break;
                case SASL_CB_PASS:
+#ifdef __APPLE__
+               case SASL_CB_OAUTH2_BEARER_TOKEN:
+#endif
                        val = ensure_password( srvc );
                        break;
                default:

This is most of the patch from this thread: https://sourceforge.net/p/isync/mailman/isync-devel/thread/20210308170245.kq4ubg24jcxezyhf%40bsd-mbp.dhcp.thefacebook.com/#msg37235467; I removed is the suppression of the warning.

I did not have to change my AuthMechs to OAUTHBEARER though, I used XOAUTH2 and it worked fine on macOS 11.6 (Big Sur). The patch will not work when compiling with cyrus-sasl-xoauth2 as SASL_CB_OAUTH2_BEARER_TOKEN is not defined.

[guibor]

Thanks a lot. So this worked!

What I did was:

1. Made the suggested changes to the source code.
2. Compile with SSL: ./configure --with-ssl=/usr/local/opt/openssl\@1.1/. This is critical because without this imap.gmail.com won't respond and using different versions/packages for SSL fails for some reason (the process starts and seems to work, but at the end of the transaction I get a weird SSL error). Don't forget to make clean before building.

[xukai92]

I also managed to make it work using the code patch (which I made avaiable at https://github.com/xukai92/isync.git).
I simply modified the formula of isync in my Homebrew (i.e. removing url,sha256 and change the Git address in head to the one above with brew edit isync) and reinstall with brew reinstall isync --build.
I didn't find it necessary to do the SSL thing as above.

For anyone just started looking into how to set up mbsync with OAUTH2, I closely followed https://cvanelteren.github.io/post/mu4e/, which lead me to the last step of failing to authenticate with mbsync using the obtained code (due to SASL issue), and then came to this issue which the solution above on how to fix that.

[Cerebus]

Updated: Ignore me, I can't even read my own config. Duplicate AuthMechs further down; removed and all is right with the world. :)

OK, I'm clearly missing something here.

• cyrus-sasl installed w/ brew
• cyrus-sasl-xoauth2 compiled and installed in cyrus (shows in pluginviewer)
• isync compiled --with-sasl and --with-ssl
• mbsync -h reports +HAVE_LIBSASL and +HAVE_LIBSSL
• AuthMechs XOAUTH2

mbsync is still trying SASL PLAIN:

Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
IMAP command 'AUTHENTICATE PLAIN <authdata>' returned an error: NO AUTHENTICATE failed.
C: 1/1  B: 0/0  F: +0/0 *0/0 #0/0  N: +0/0 *0/0 #0/0

[bmclean2]

Successfully got this to work using the above code patch. I needed to use ./configure --with-ssl=/usr/local/opt/openssl\@1.1/

Of note: When I run mbsync, I get the following warning on the oauth2 accounts: Warning: SASL wants more steps despite successful IMAP authentication. Ignoring... Anyone else? I'm guessing that this is due to some peculiarity in Apple's SASL. I'm running Mojave.

[jngeist]

Did you compile mbsync --with-sasl=/path/to/cyrus? It compiles and no longer calls the Apple provided implementation, but I wasn't able to get it to work. Running pluginviewer only shows EXTERNAL.

It wasn't immediately clear to me why this was the only plugin listed, so I ended up patching mbsync. By adding the above mentioned callback, I was able to get mbsync working with the Apple provided implementation. Certainly not ideal, but at least it works until I can figure something else out.

I can't speak for you (much less the you of a year ago), but I think the issue is that the ./configure invocation for this package misses something crucial on macos which can cause it to fail in a way that is very easy to overlook. Specifically, on macos trying to install cyrus-sasl-xoauth2 as described into the docs will attempt to install the plugin to the system implementation of sasl, which will fail, even if cyrus-sasl is properly installed.

The solution to this is to direct it to the correct target: ./configure --with-cyrus-sasl=/path/to/cyrus-sasl. Then, make install will actually install the plugin for cyrus-sasl, after which everything should start working properly. (Given all the other givens, like mbsync being compiled to use cyrus-sasl, etc.) Rather than having to hack mbsync to get it to play nicely with the system sasl implementation, this seems to solve the problem using cyrus-sasl and cyrus-sasl-xoauth2, which is what we're nominally here for.

This might be worth noting in the readme for macos users, given the number of us bumbling around here in the issues. :)