[Glen Eustace]

I have just installed OTOBO 10.1 using the docker instructions as my standalone deployment has been having issues with parsing text/quoted-plain email messages which I have been unable to resolve. The only deviation from the instructions was to not run the db container but instead to connect to the existing database.  I achieved this by using a socat relay on the docker host.

Issue 1.

Fetchmail is not being run, I am having to run it manually from the Admin>Postmaster Mail Accounts.

Issue 2.

Daemon container cant connect to the ElasticSearch, I have tried including the error message but WordFence keeps blocking it, I am happy to email instead but the message says that there is an error 500 whilst attempting to do a post to localhost:9200 to the controller /ticket/_delete_by_query

ERROR: OTOBO-otobo.Daemon.pl – Daemon Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker-10
Perl: 5.36.0 OS: linux Time: Tue Aug 23 08:20:11 2022

Traceback (12077):
Module: Kernel::GenericInterface::ErrorHandling::HandleError Line: 115
Module: Kernel::GenericInterface::Requester::_HandleError Line: 543
Module: Kernel::GenericInterface::Requester::Run Line: 340
Module: Kernel::GenericInterface::Event::Handler::Run Line: 214
Module: (eval) Line: 232
Module: Kernel::System::EventHandler::EventHandler Line: 226
Module: Kernel::System::EventHandler::EventHandlerTransaction Line: 279
Module: Kernel::System::ObjectManager::ObjectEventsHandle Line: 441
Module: Kernel::System::Daemon::DaemonModules::SchedulerTaskWorker::Run Line: 246
Module: (eval) Line: 386
Module: main::Start Line: 386
Module: bin/otobo.Daemon.pl Line: 164

[Glen Eustace]

This is weird. If I run the Console command, it says there are no configured mail accounts, but if I use the web UI, and click on ‚Fetch Mail‘, it works. If I look in the db directly, there is an entry in the mail_accounts table.

otobo@997c99e8bd0b:~/Kernel$ ~/bin/otobo.Console.pl Maint::PostMaster::MailAccountFetch

No configured mail accounts found!

 

[bes]

Hi Glen,

I have no idea about the mail configuration. But I noticed something in the Elasticsearch error message. The message says that the Daemon wants to talk to localhost:9200 . But in a regular Docker-based setup ES is running on elastic:9200. This ES configuration can be adapted in the Admin-Interface for „Webservices“.

In new installations this should not happen as the initial insert into the database is changed in the Docker image. See https://github.com/RotherOSS/otobo/blob/rel-10_1/otobo.web.dockerfile#L114 . Did you migrate data from your old installation into the new installation?

Best regards,

Bernhard

 

[Glen Eustace]

This is the regular docker install, but I guess that I might have overwritten the config as I used my existing database.

I didn’t migrate the data, I simple attached the database by updating the Kernel>Config.pm

You are 100% correct, the value for the container is wrong because that was how I had it in my original deployment.  I am struggling to work out how to fix it as the field in the database is a blob. I cant find where in the System Config I can fix this.

[Glen Eustace]

OK, I think I worked that one out phpMyAdmin allows me to edit the blob by uploading a file.

[Glen Eustace]

Hmmm, that didn’t work.  It still is using localhost. I have tried rebuilding and sync’ing the config and stop and restarting the daemon.  No luck so far.

[Glen Eustace]

Well I don’t know what I have done but the ElasticSearch is now working, I am using the console.pl script Maint::Elasticsearch::Migration to load the history. 30% of tickets so far but looks good.

Running Maint::PostMaster::MailAccountFetch also now works. But I haven’t done anything that I am aware of that might have had an impact there.

BUT, the web interface no longer works at all.  I cant seem to find any error logs anywhere that give a clue as to what is happening. The URL just times out

[Glen Eustace]

Solved, our fail2ban service had put my IP address into our boundary firewall.  Haven’t worked out why yet but at least I have access again. So, mail working again, elasticsearch not generating errors – progress.  At one point I had an input field at the top of the agent screen labelled elastic search.  This has also vanished so I am now on the hist to try to work out how to get that back again.

[Glen Eustace]

Got that one nailed now too. Now hopefully with a functional system will see if has cured the issue with un-parsable emails.

[bes]

Hi Glen,

glad to hear that your installation  is now in a workable state.

Using a database, which is not running under Docker, from a Docker-based web server is a valid use case. That the Elasticsearch Webservice Konfiguration must then be adapted manually is understandable. It’s hard to automate that, as many different setups are possible. But there shouldn’t have been a need to update the webservice configuration in the database directly. This should have been possible from the Admin-Interface.

I’m also curious about the socat relay. Was that necessary because the database can only be reached via an SSH-tunnel? My understanding was that the services running in Docker container can access the same network resources as are reachable from the Docker host.

Also, did you edit _docker-compose/otobo-base.yml_ in order to deactivate the Docker-based database service? I had been thinking about moving the service db into a dedicated configuration file. This would all to suppress the service by only editing the _.env_ file.

Best regards,

Bernhard

 

[Glen Eustace]

Finally getting to the bottom of the Email Parsing issue.  There seems to be an issue with MariaDB and collation.  I get the following error;

DBD::mysql::db do failed: Incorrect string value: ‚\xF0\x9F\x98\x8A\x0A\x0A…‘ for column otobo.article_data_mime.a_body at row 1 at /opt/otobo/Kernel/System/DB.pm line 556.
ERROR: OTOBO-otobo.Console.pl-Maint::PostMaster::Read-10 Perl: 5.36.0 OS: linux Time: Mon Aug 29 23:21:07 2022

Message: Incorrect string value: ‚\xF0\x9F\x98\x8A\x0A\x0A…‘ for column otobo.article_data_mime.a_body at row 1, SQL: ‚
INSERT INTO article_data_mime (
article_id, a_from, a_reply_to, a_to, a_cc, a_bcc, a_subject, a_message_id,
a_message_id_md5, a_in_reply_to, a_references, a_content_type, a_body,
incoming_time, content_path, create_time, create_by, change_time, change_by)
VALUES (
?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ‚2022-08-29 23:21:07‘, ?, ‚2022-08-29 23:21:07‘, ?
)

But the collation appears to be correct – it is currently utf8-general-ci, maybe its supposed to be mtf8mb4

[Glen Eustace]

Yep, finally got there.  The DB and all the tables need to be utf8mb4 and utf8mb4-general-ci.  So converting to the docker version was probably a waste of time but did have the benefit of learning more stuff.

 

[bes]

Yes, the broken utf8 character set of MySQL is really annoying. It does not allow to store characters that are encoded by four octets. An example for such an character is a simple smiley:

bes:~ $ uni -8 1F601
😁 - U+1F601 - F0 9F 98 81 - GRINNING FACE WITH SMILING EYES

The fix is to check the character set of the OTOBO tables and of the columns of the tables. The character set must be utf8mb4. The mb4 stands for four bytes. See also https://github.com/RotherOSS/otobo/issues/1879 .

 

[Autor]

Beiträge