Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: DBMail: users

Fwd: Fwd: dbmail3: still some strange POP3 troubles

 

 

DBMail users RSS feed   Index | Next | Previous | View Threaded


h.reindl at thelounge

Feb 22, 2012, 6:33 AM

Post #1 of 7 (583 views)
Permalink
Fwd: Fwd: dbmail3: still some strange POP3 troubles

to make it even more strange

my webbackend counts 1136 messages in 4 folders (database)
this is normally exactly right and counts really all

webmail over imap says 2424 in "inbox"
google says it received 200 messages via POP3, still 2224 outstanding
google webinterface shows 110 messages in inbox
thunderbird over IMAP says 245 in the inbox

can someone blow me a bullet in the head or explain this differences?

-------- Original-Nachricht --------
Betreff: [Dbmail] Fwd: dbmail3: still some strange POP3 troubles
Datum: Wed, 22 Feb 2012 15:06:24 +0100
Von: Reindl Harald <h.reindl [at] thelounge>
Antwort an: DBMail mailinglist <dbmail [at] dbmail>
Organisation: the lounge interactive design
An: Mailing-List dbmail <dbmail [at] dbmail>

gmail tells there are 2244 messages still on the server and 200 received
webmail says in the inbox are 2424 messages


2. Februar 2012 14:54 Fehler: Der Server ist vorübergehend nicht verfügbar.
Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
22. Februar 2012 14:51 Fehler: Der Server ist vorübergehend nicht verfügbar.
Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
22. Februar 2012 14:47 200 E-Mails wurden abgerufen. 2224 weitere Nachrichten.
22. Februar 2012 14:45 Fehler: Der Server ist vorübergehend nicht verfügbar.
Der Server hat die Fehlermeldung "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."
ausgegeben.

-------- Original-Nachricht --------
Betreff: [Dbmail] dbmail3: still some strange POP3 troubles
Datum: Wed, 22 Feb 2012 14:52:50 +0100
Von: Reindl Harald <h.reindl [at] thelounge>
Antwort an: DBMail mailinglist <dbmail [at] dbmail>
Organisation: the lounge interactive design
An: Mailing-List dbmail <dbmail [at] dbmail>

it drives me crazy :-(

gmail accessing over POP3 a ssh-forward of the reported
problem account: "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."

a few minutes ago another customer said outlook is stop fetching
mail ona large message with 8.3 MB over POP3, move the message
via webmail in another folder -> all ok, receive other mails

two hours ago another customer with AppleMail complained he does
not receive any new messages since the upgrade and also did
only see our update-announcement in horde-webmail while the
log shows successful pop3-logins



--

Mit besten Grüßen, Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm



--

Mit besten Grüßen, Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm
Attachments: Nachrichtenteil als Anhang (0.14 KB)
  signature.asc (0.26 KB)


paul at nfg

Feb 22, 2012, 7:06 AM

Post #2 of 7 (552 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

On 02/22/2012 03:33 PM, Reindl Harald wrote:
> to make it even more strange
>
> my webbackend counts 1136 messages in 4 folders (database)
> this is normally exactly right and counts really all

How?


> webmail over imap says 2424 in "inbox"
> google says it received 200 messages via POP3, still 2224 outstanding

At least webmail and google are consistent about the numbers.

Maybe this is how google works: fetch batches of 200.

> google webinterface shows 110 messages in inbox

What do you mean? How is the google webinterface connected to dbmail?

> thunderbird over IMAP says 245 in the inbox

Try a raw IMAP connection:

~$ nc dbmail imap
* OK [CAPABILITY IMAP4rev1 AUTH=LOGIN STARTTLS ID] dbmail 3.0.0 ready.
x login testuser1 test
x OK [.CAPABILITY IMAP4rev1 ACL RIGHTS=texk NAMESPACE CHILDREN SORT QUOTA
THREAD=ORDEREDSUBJECT UNSELECT IDLE STARTTLS ID] User testuser1
authenticated
x select INBOX
* 7 EXISTS
* 0 RECENT
* FLAGS (\Seen \Answered \Deleted \Flagged \Draft 2 t Tést tës)
* OK [.PERMANENTFLAGS (\Seen \Answered \Deleted \Flagged \Draft 2 t Tést
tës \*)] Flags allowed.
* OK [UIDNEXT 3075781] Predicted next UID
* OK [UIDVALIDITY 3022] UID value
* OK [UNSEEN 1] first unseen message
x OK [READ-WRITE] SELECT completed

The EXISTS response is what IMAP says is the number of messages in the
selected mailbox.


Same for POP3
~$ nc dbmail pop3
+OK DBMAIL pop3 server ready to rock
<832a5a2a60983c51199127a8329ca6c7 [at] volume-0->
user testuser1
+OK Password required for testuser1
pass test
+OK testuser1 has 7 messages (92422666 octets)
list
+OK 7 messages (92422666 octets)
1 1615
2 601
3 70
4 5
5 5
6 92420252
7 118
..


Same number of messages as with IMAP.


> can someone blow me a bullet in the head or explain this differences?

But I managed to reproduce the RETR error. Will trace it myself.



>
> -------- Original-Nachricht --------
> Betreff: [Dbmail] Fwd: dbmail3: still some strange POP3 troubles
> Datum: Wed, 22 Feb 2012 15:06:24 +0100
> Von: Reindl Harald <h.reindl [at] thelounge>
> Antwort an: DBMail mailinglist <dbmail [at] dbmail>
> Organisation: the lounge interactive design
> An: Mailing-List dbmail <dbmail [at] dbmail>
>
> gmail tells there are 2244 messages still on the server and 200 received
> webmail says in the inbox are 2424 messages
>
>
> 2. Februar 2012 14:54 Fehler: Der Server ist vorübergehend nicht verfügbar.
> Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
> 22. Februar 2012 14:51 Fehler: Der Server ist vorübergehend nicht verfügbar.
> Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
> 22. Februar 2012 14:47 200 E-Mails wurden abgerufen. 2224 weitere Nachrichten.
> 22. Februar 2012 14:45 Fehler: Der Server ist vorübergehend nicht verfügbar.
> Der Server hat die Fehlermeldung "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."
> ausgegeben.
>
> -------- Original-Nachricht --------
> Betreff: [Dbmail] dbmail3: still some strange POP3 troubles
> Datum: Wed, 22 Feb 2012 14:52:50 +0100
> Von: Reindl Harald <h.reindl [at] thelounge>
> Antwort an: DBMail mailinglist <dbmail [at] dbmail>
> Organisation: the lounge interactive design
> An: Mailing-List dbmail <dbmail [at] dbmail>
>
> it drives me crazy :-(
>
> gmail accessing over POP3 a ssh-forward of the reported
> problem account: "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."
>
> a few minutes ago another customer said outlook is stop fetching
> mail ona large message with 8.3 MB over POP3, move the message
> via webmail in another folder -> all ok, receive other mails




> two hours ago another customer with AppleMail complained he does
> not receive any new messages since the upgrade and also did
> only see our update-announcement in horde-webmail while the
> log shows successful pop3-logins


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


paul at nfg

Feb 22, 2012, 7:35 AM

Post #3 of 7 (549 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

Seems like the TOP command was broken, not RETR.

I've just pushed a fix for that. GMail now is happy again here.

--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


h.reindl at thelounge

Feb 22, 2012, 7:49 AM

Post #4 of 7 (551 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

info: 3.0.1 HEAD currently, same results
http://git.dbmail.eu/paul/dbmail/commit/?id=24901216f5d51391e96e46500613c4e41e30c36f

below my count_mails() and RAW IMAP/POP3 results

thanks for your feddback/help/debugging/work -> anything!

Am 22.02.2012 16:06, schrieb Paul J Stevens:
> On 02/22/2012 03:33 PM, Reindl Harald wrote:
>> to make it even more strange
>>
>> my webbackend counts 1136 messages in 4 folders (database)
>> this is normally exactly right and counts really all
>
> How?

well, with manual queries seems that there a hughe amount of messages have
"deleted_flag", maybe they started cleanup per webmail in the meantime

i am not permitet to contact the user directly and have to do all communication
via a non-tech "communications coordinator" not understanding any word i say/write
which makes my debugging as hard as possible

/**
* (PHP)
* count messages of a user
*
* @param integer $user
* @return integer
* @access public
*/
public function count_mails($user_idnr)
{
$row = $this->db->fetch_row($this->db->query('select count(message_idnr) from `dbmail_mailboxes`,
`dbmail_messages` where owner_idnr=' . (int)$user_idnr . ' and
dbmail_messages.mailbox_idnr=dbmail_mailboxes.mailbox_idnr and (dbmail_messages.deleted_flag=0 and
dbmail_messages.status<2);'));
return $row[0];
}

dbmail_messages.mailbox_idnr=dbmail_mailboxes.mailbox_idnr and (dbmail_messages.deleted_flag=0 and
dbmail_messages.status<2);

+---------------------+

| count(message_idnr) |

+---------------------+
| 1138 |
+---------------------+
1 row in set (0.01 sec)

dbmail_messages.mailbox_idnr=dbmail_mailboxes.mailbox_idnr and (dbmail_messages.deleted_flag=0);+---------------------+
| count(message_idnr) |
+---------------------+
| 1138 |
+---------------------+
1 row in set (0.00 sec)

dbmail_messages.mailbox_idnr=dbmail_mailboxes.mailbox_idnr;
+---------------------+
| count(message_idnr) |
+---------------------+
| 2454 |
+---------------------+
1 row in set (0.00 sec)

>> webmail over imap says 2424 in "inbox"
>> google says it received 200 messages via POP3, still 2224 outstanding
>
> At least webmail and google are consistent about the numbers.

but it is wrong, surely wrong

> Maybe this is how google works: fetch batches of 200.
>> google webinterface shows 110 messages in inbox
>
> What do you mean? How is the google webinterface connected to dbmail?

this is the "fetch account via pop3 to google-account" thing
so it says it fecthed 200 mails via POP3 until error happended
it shows only via https://mail.google.com/ in the inbox

my thunderbird is connected to the same gmail account
via IMAP and shows 245 messages

> Try a raw IMAP connection:
> ~$ nc dbmail imap
> * OK [CAPABILITY IMAP4rev1 AUTH=LOGIN STARTTLS ID] dbmail 3.0.0 ready.
> x login testuser1 test
> x OK [.CAPABILITY IMAP4rev1 ACL RIGHTS=texk NAMESPACE CHILDREN SORT QUOTA
> THREAD=ORDEREDSUBJECT UNSELECT IDLE STARTTLS ID] User testuser1
> authenticated
> x select INBOX
> * 7 EXISTS
> * 0 RECENT

x select INBOX
* 2426 EXISTS
* 2 RECENT

> Same for POP3
> ~$ nc dbmail pop3
> +OK DBMAIL pop3 server ready to rock
> <832a5a2a60983c51199127a8329ca6c7 [at] volume-0->
> user testuser1
> +OK Password required for testuser1
> pass test
> +OK testuser1 has 7 messages (92422666 octets)
> list
> +OK 7 messages (92422666 octets)

+OK <3c7e.39e.4f450c1f.kSpuDsPBfNM5i3mFqWjG0A==@caladan.thelounge.net>
user **disclosed**
+OK
pass **disclosed**
+OK **disclosed** has 2426 messages (359588826 octets)

so well, IMAP/POP3 are showing the same count
but why a different than my query?

> But I managed to reproduce the RETR error. Will trace it myself.

i guess this is the source of most troubles except lmtpd
i can NOT reproduce the lmtpd-crash running gdb with 3.0.1
but the "normal" service is still crashing :-(

>> -------- Original-Nachricht --------
>> Betreff: [Dbmail] Fwd: dbmail3: still some strange POP3 troubles
>> Datum: Wed, 22 Feb 2012 15:06:24 +0100
>> Von: Reindl Harald <h.reindl [at] thelounge>
>> Antwort an: DBMail mailinglist <dbmail [at] dbmail>
>> Organisation: the lounge interactive design
>> An: Mailing-List dbmail <dbmail [at] dbmail>
>>
>> gmail tells there are 2244 messages still on the server and 200 received
>> webmail says in the inbox are 2424 messages
>>
>>
>> 2. Februar 2012 14:54 Fehler: Der Server ist vorübergehend nicht verfügbar.
>> Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
>> 22. Februar 2012 14:51 Fehler: Der Server ist vorübergehend nicht verfügbar.
>> Der Server hat die Fehlermeldung "Error in RETR command: >" ausgegeben.
>> 22. Februar 2012 14:47 200 E-Mails wurden abgerufen. 2224 weitere Nachrichten.
>> 22. Februar 2012 14:45 Fehler: Der Server ist vorübergehend nicht verfügbar.
>> Der Server hat die Fehlermeldung "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."
>> ausgegeben.
>>
>> -------- Original-Nachricht --------
>> Betreff: [Dbmail] dbmail3: still some strange POP3 troubles
>> Datum: Wed, 22 Feb 2012 14:52:50 +0100
>> Von: Reindl Harald <h.reindl [at] thelounge>
>> Antwort an: DBMail mailinglist <dbmail [at] dbmail>
>> Organisation: the lounge interactive design
>> An: Mailing-List dbmail <dbmail [at] dbmail>
>>
>> it drives me crazy :-(
>>
>> gmail accessing over POP3 a ssh-forward of the reported
>> problem account: "Error in RETR command: =E4=E9=E0 =EE=E0=E5=F9=F8=FA =EC=F7=E1=EC =E0=FA =E4=E..."
>>
>> a few minutes ago another customer said outlook is stop fetching
>> mail ona large message with 8.3 MB over POP3, move the message
>> via webmail in another folder -> all ok, receive other mails
>
>> two hours ago another customer with AppleMail complained he does
>> not receive any new messages since the upgrade and also did
>> only see our update-announcement in horde-webmail while the
>> log shows successful pop3-logins

--

Mit besten Grüßen, Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm
Attachments: signature.asc (0.26 KB)


h.reindl at thelounge

Feb 22, 2012, 8:10 AM

Post #5 of 7 (551 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

Am 22.02.2012 16:35, schrieb Paul J Stevens:
> Seems like the TOP command was broken, not RETR.
>
> I've just pushed a fix for that. GMail now is happy again here.

hallelujha - gmail does no longer complain
maybe this does solve all sort of other
POP3 problems with one strike

i will see.....

THANK YOU SOU MUCH!

--

Reindl Harald
the lounge interactive design GmbH
A-1060 Vienna, Hofmühlgasse 17
CTO / software-development / cms-solutions
p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40
icq: 154546673, http://www.thelounge.net/

http://www.thelounge.net/signature.asc.what.htm
Attachments: signature.asc (0.26 KB)


paul at nfg

Feb 22, 2012, 8:13 AM

Post #6 of 7 (557 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

Harry,

On 02/22/2012 04:49 PM, Reindl Harald wrote:
> info: 3.0.1 HEAD currently, same results
> http://git.dbmail.eu/paul/dbmail/commit/?id=24901216f5d51391e96e46500613c4e41e30c36f


You need

http://git.dbmail.eu/paul/dbmail/commit/?id=8e4f4ae008614912fc1cbf5c2b4eea57978150db

for the gmail fix.

It was done after bumping the version.


> well, with manual queries seems that there a hughe amount of messages have
> "deleted_flag", maybe they started cleanup per webmail in the meantime

Probably.

> mysql> select count(message_idnr) from `dbmail_mailboxes`, `dbmail_messages` where owner_idnr=113 and
> dbmail_messages.mailbox_idnr=dbmail_mailboxes.mailbox_idnr and (dbmail_messages.deleted_flag=0 and
> dbmail_messages.status<2);

You really shouldn't ignore messages with deleted_flag=1. Those are
marked for deletion in IMAP, but are still available both in IMAP and
POP3. Only after the IMAP client issues a EXPUNGE command are they
removed (status=2) and queued for purging from the database.

>>> webmail over imap says 2424 in "inbox"
>>> google says it received 200 messages via POP3, still 2224 outstanding
>>
>> At least webmail and google are consistent about the numbers.
>
> but it is wrong, surely wrong

No it's not.

>> Maybe this is how google works: fetch batches of 200.
>>> google webinterface shows 110 messages in inbox
>>
>> What do you mean? How is the google webinterface connected to dbmail?
>
> this is the "fetch account via pop3 to google-account" thing
> so it says it fecthed 200 mails via POP3 until error happended
> it shows only via https://mail.google.com/ in the inbox
>
> my thunderbird is connected to the same gmail account
> via IMAP and shows 245 messages

Then thunderbird is not reading the IMAP results.

> so well, IMAP/POP3 are showing the same count
> but why a different than my query?

Your query is incorrect.

>> But I managed to reproduce the RETR error. Will trace it myself.
>
> i guess this is the source of most troubles except lmtpd
> i can NOT reproduce the lmtpd-crash running gdb with 3.0.1
> but the "normal" service is still crashing :-(

So maybe it's not the message itself, but the way postfix talks to
dbmail - just guessing here.


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail


paul at nfg

Feb 22, 2012, 8:14 AM

Post #7 of 7 (548 views)
Permalink
Re: Fwd: Fwd: dbmail3: still some strange POP3 troubles [In reply to]

On 02/22/2012 05:10 PM, Reindl Harald wrote:
>
>
> Am 22.02.2012 16:35, schrieb Paul J Stevens:
>> Seems like the TOP command was broken, not RETR.
>>
>> I've just pushed a fix for that. GMail now is happy again here.
>
> hallelujha - gmail does no longer complain
> maybe this does solve all sort of other
> POP3 problems with one strike
>
> i will see.....
>
> THANK YOU SOU MUCH!

Phfwww. Time to go home and hug the kids.


--
________________________________________________________________
Paul J Stevens pjstevns @ gmail, twitter, skype, linkedin

* Premium Hosting Services and Web Application Consultancy *

www.nfg.nl/info [at] nfg/+31.85.877.99.97
________________________________________________________________
_______________________________________________
DBmail mailing list
DBmail [at] dbmail
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

DBMail users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.