Reference from FAQ/Help under "Syntax":

Want only the owner of the record and those with admin permissions to see a field(s)

Use:
|;
if (($db_userid eq $rec{'UserID'}) || ($per_admin)) {
print qq| [The record information] |; <BR>
}
print qq|

So in your case try using:

|;
if (($db_userid eq $rec{'UserID'}) || ($per_admin)) {

print " <img border=0 src=$images_url/a_sep.gif>
<A HREF=$db_script_link_url&modify_form_record=1&modify=$rec{'ID'}><img border=0 src=$images_url/aedit.gif alt=Edit></a> " if ($per_mod);
}
print qq|

This would allow only the record holder and admin to view the links.

Unoffical DBMan FAQ
http://webmagic.hypermart.net/dbman/