https://www.zabbix.com/documentation/3.4/manual/config/items/itemtypes/jmx_monitoring says that:

1. A backslash character should be escaped as well.
2. If the object name or attribute name contains spaces or commas double-quote it

1st statemet has many exceptions.

I have this real ObjectName:

com.example:type="metrics.akka.systems.ClusterSystem.dispatchers.akka_profile-dispatcher.actors._user_backendProxyprofile_\*.mailbox-size"

Yes, it contains quoted part.
Yes, it contains star.
Yes, the star is escaped by backslash.
A note: practical tests show that according to Java itself - star in object name must be escaped and they all must be quoted, otherwise java app cannot be compiled or started.

Attribute name in this case is simple word, we omit it for now.

Working JMX key for this Mbean is:

jmx[com.example:type="metrics.akka.systems.ClusterSystem.dispatchers.akka_profile-dispatcher.actors._user_backendProxyprofile_\*.mailbox-size",simple]

So, to monitor such Mbean, I have to ignore note on the documentation about quoting and for escaping.

Let's get back to escaping in Attribute:

Practical tests show that successful result is when number of backslashes in attribute:

Yes, single backslash may stay unescaped, but 2 backslashes should be written in key as 3 or 4 backslashes.
In other words, backslashes in Attribute should be escaped, but allow to put one baskslash less. Yes - it sounds bad, but it is so.

So, documentation requirement for escaping is incorrect in general. It should be separated for ObjectName and Attribute.
Or ... maybe something should be fixed.

Some details as for response:
If request contains odd number of slashes in Attribute - response error contains the number+1 of slashes as reference of missing Mbean, which is strange.
... if even number of slashes - response error contains the correct number of backslashes.

My answer for 2nd doc statement:
Practical test shows that it's not really required to quote values with spaces, key jmx[com.example:type=A B,s p] works just fine.

And at the end, notes just in case:
comma inside ObjectName requires ObjectName to be quoted in java itself and the comma causes that ObjectName in key should be quoted.
right square bracket ] in ObjectName may be used as is, without any escaping or quoting.

All tests performed using zabbix 3.4.4