Some bugs and a doubt

1/ The bot is unable to detect URLs if I put the URL in a paragraph. Let's take two examples.

Example 1:
If I directly send an URL(https://en.m.wikipedia.org/wiki/Lion) in a channel it will detect the URL and will do a domain and URL test, which is nice.

Example 2:
But if I send the same URL like this:

This is the Wikipedia page of Lion.https://en.m.wikipedia.org/wiki/Lion

The the bot is unable to detect the URL and sends a error message like this.

2/ I noticed a very rare bug. If I put an URL once in the channel the bot gives me results 2 times. It doesn't happen often but it happens very rarely. I attached a picture below showing the bug.

3/ Last a doubt. I added some extensions for the bot which it wants to ignore. Do you think it's enough or I need to add more for files like .pdf, .txt etc?

[ ##Image formats to ignore##
'image/gif','image/jpeg','image/png','image/svg','image/apng','image/avif','image/webp',


##Audio formats to ignore##
'audio/mpeg','audio/wave'
,'audio/ogg','audio/mp3','audio/webm',


##Video formats to ignore##
'video/mp4','video/webm','video/mkv','video/flv','video/vob','video/ogg','video/ogv','video/gif','video/avi','video/wmv','video/3gp']

[The-Bow-Hunter]

To your second example: for some reason it thinks Lion.https would be a valid domain already. Scinece it isnt Virustotal will throw an error stating it is invalid. I might have to work on the regex for that one

[The-Bow-Hunter]

To that bug: it was because you sended the second one to eraly. One of the messages was still regarding the first Link. It will mostly do an URL and an Domain check (so if it likes wikipedia and if it likes that specific page), but the time they need is different

[The-Bow-Hunter]

pdf would be application/pdf

[The-Bow-Hunter]

Just a quick update to your 3. message: There have been some new problems with pdfs regarding their security... https://helpx.adobe.com/security/products/acrobat/apsb21-29.html

Thanks for the update I removed application/pdf from the ignored file list in the bot file. This was really helpful.
And did you find a way to update the URL finding regex so that it's easy for the bot to find URLs in paragraphs and sentences.

I found a new URL regex.

(http|ftp|https)://([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:/~+#-]*[\w@?^=%&/~+#-])?

You can check it at https://regex101.com/

Or run this code

# coding=utf8
# the above tag defines encoding for this document and is for Python 2.x compatibility

import re

regex = r"(http|ftp|https)://([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:/~+#-]*[\w@?^=%&/~+#-])?"

test_str = ("https://taaku18.github.io/modmail/getting-started/ \n"
	"https://regex101.com/\n"
	"https://en.m.wikipedia.org/wiki/Lion\n"
	"This  is Wikipedia page of lion.https://en.m.wikipedia.org/wiki/Lion \n"
	"http://yahoo.com \n"
	" ")

matches = re.finditer(regex, test_str, re.MULTILINE | re.IGNORECASE)

for matchNum, match in enumerate(matches, start=1):
    
    print ("Match {matchNum} was found at {start}-{end}: {match}".format(matchNum = matchNum, start = match.start(), end = match.end(), match = match.group()))
    
    for groupNum in range(0, len(match.groups())):
        groupNum = groupNum + 1
        
        print ("Group {groupNum} found at {start}-{end}: {group}".format(groupNum = groupNum, start = match.start(groupNum), end = match.end(groupNum), group = match.group(groupNum)))

# Note: for Python 2.7 compatibility, use ur"" to prefix the regex and u"" to prefix the test string and substitution.

To that bug: it was because you sended the second one to eraly. One of the messages was still regarding the first Link. It will mostly do an URL and an Domain check (so if it likes wikipedia and if it likes that specific page), but the time they need is different

The thing is I only sended one URL but the bot gives me total 4 embeds [2 URL check 2 domain check]. I never sent 2 URLs back to back. Like I said earlier it happens very rarely. For some reason the bot sends 4 embeds for 1 url

[The-Bow-Hunter]

sry that i needed taht long to reply. I am pretty busy with university at the moment, i will take a look at it once i have time