Add more comic strips

[ArtskydJ]

Before you write a scraper for comicsrss, please know that I don't want comicsrss to have some types of comic strips.

I don't want comicsrss to have sexually-suggestive comics. For example, I've considered killing the rss feed for 9 Chickweek Lane, and I still might kill it someday. I'm going to avoid adding anything to comicsrss that's more suggestive than that.

I might kill off political comics. I haven't yet, but I've been strongly considering it for a while now. Internet politics discussions tend to be tribal and echo-chambery, but political comics step that up a few notches.

---

List of comic strips/websites that folks have requested, and who requested them

Planned:

• Dilbert http://dilbert.com/
• Arcamax https://www.arcamax.com/comics
• Comics Kingdom https://www.comicskingdom.com/
• Creators.com https://www.creators.com/categories/comics
  • Mike P (email) - Spectickles
• The Far Side https://www.thefarside.com
  • Brian W (email)
  • Coleman (email)
  • Mike B (patreon)
• Ctrl+Alt+Delete https://cad-comic.com/feed/
  • Milan A (email)

I'd love to see some Comics Kingdom strips added, if possible. (For me, personally, mainly Bizarro, Rhymes with Orange, and Darrin Bell.)

[ArtskydJ]

Arcamax has Bizarro, Dilbert, and Rhymes with Orange, and Darrin Bell.

Both Comics Kingdom, and Arcamax look like they will be much more difficult to scrape than gocomics.

[ArtskydJ]

Added Dilbert today.

[ArtskydJ]

I don't remember why I thought Arcamax would be particularly difficult. It doesn't look like it will be that hard...

<a class="prev" href="/thefunnies/brilliantmindofedisonlee/s-2160999" title="Brilliant Mind of Edison Lee 1/3/2019"><span class="entypo-left-open"></span></a>
  <span class="cur">January  4</span>
<a class="next-off" href="#"><span class="entypo-right-open"></span></a>

<!-- ... -->

<figure class="comic">
  <img id="comic-zoom" data-zoom-image="/newspics/168/16885/1688589.gif" src="/newspics/168/16885/1688589.gif"  data-width="600" data-height="187" alt="" class="img-responsive the-comic" title="click or tap to zoom" />
  <cite class="comic-copyright">(c) 2019 John Hambrock.  Dist. by King Features Syndicate, Inc.</cite>
</figure>

Hopefully I'll get around to it within a few weeks.

[infinitytec]

Could I request Sherman's Lagoon and Freefall (the latter is a webcomic found at freefall.purrsia.com)?

[ArtskydJ]

Sherman's lagoon is on Comics Kingdom. If/when I add comics Kingdom, I can @ you in this thread.

I doubt I'll add Freefall unless it is part of a larger site like Comics Kingdom or Arcamax. If there's enough demand for it, I might add it.

Or you could look into adding it similar to dilbert was added:
https://github.com/ArtskydJ/comicsrss.com/blob/gh-pages/_generator/scraper-dilbert/index.js
There isn't really an API for making a scraper... ☹️

---

This is what I did for dilbert (and the process would be similar on freefall):

1. Grab a page that shows multiple comics, including the latest comic
   a. For dilbert it was https://dilbert.com
   b. For freefall it might be http://freefall.purrsia.com/lastthree.htm
2. Parse the HTML to turn it into an array like this:

[
    {
        "titleAuthorDate": "Freefall by Tugrik for Wednesday 6/12/2019",
        "url": "http://freefall.purrsia.com/ff3300/fc03290.htm",
        "date": "2019-06-12",
        "comicImageUrl": "http://freefall.purrsia.com/ff3300/fc03290.png"
    },
    ...
]

3. Open the cached version of that array, and merge them together. (If I don't have the latest comic in the cached array, then I need to push it onto the array.)
4. Write the cached file to disk.
5. Integrate it with the rest of the system. (If you do everything else I would be more than happy to integrate your scraper.)

[infinitytec]

Thanks for the information! I'll look into it and see what I can do!

[ArtskydJ]

I made an API and published it in the README.

[jgbishop]

Any progress on this? I've looked into scraping Comics Kingdom in the past year myself, and it's pretty difficult. Lots of the page gets loaded dynamically when first visited in a web browser. The publishers are clearly trying their best to prevent scraping, but my scraping knowledge is fairly limited when it comes to dynamic data. Maybe the arcamax website would be easier?