Port to WebKit2/webkitgtk-4.0 about astroid HOT 31 CLOSED

Upvoting this issue: 👍

Homebrew on macOS only supports newer webkitgtk:

$ brew info webkitgtk
webkitgtk: stable 2.10.9 (bottled)
Full-featured Gtk+ port of the WebKit rendering engine
http://webkitgtk.org
/usr/local/Cellar/webkitgtk/2.10.9_1 (331 files, 40.2M) *
  Poured from bottle on 2016-06-22 at 12:36:40
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/webkitgtk.rb
==> Dependencies
Build: xz ✔, cmake ✔
Required: gtk+3 ✔, libsoup ✔, enchant ✔, webp ✔

Two questions:

1. What version of webkitgtk does this translate to in GNU Linux/Debian packaging speak? I'm always confused by the package naming conventions... (see this stackoverflow thread)
2. Should the method for version checking webkitgtk in scons be updated to reflect the library name in macOS? See:

$ scons --prefix=/usr/local build
scons: Reading SConscript files ...
building version v0.5-228-g31ad8c35 (git)..
debug flag enabled: True
Checking for pkg-config... yes
Checking for gtkmm-3.0 >= 3.10... yes
Checking for glibmm-2.4... yes
Checking for gmime-2.6 >= 2.6.18... yes
Checking for webkitgtk-3.0... no
webkitgtk not found.

from astroid.

Can netsurf be embedded and used as GTK widget?

While I prefer a smaller / lightweight solution I am not necessarily
convinced it will be safer.

It seems that several of the lightweight solutions are not completely
done yet, which is another point for using a more mainstream engine: it
is sure to get security updates, it is sure to be feature complete, and
it will remain an active project for the foreseeable future.

Given that manpower is a finite resource for both astroid and the webkit
projects - and web browsers are complex stuff - I am still leaning
towards webkit2 being the most pragmatic, and possible best,
alternative.

WebKit2 does have a DOM interface which matches the one from WebKit1,
this makes porint easy - except it is not accessible from the main
thread. A web extension must be written (this will run in the
webkit-process) which communicates with the astroid-process [0].

[0] https://blogs.igalia.com/carlosgc/2013/09/10/webkit2gtk-web-process-extensions/

from astroid.

Great stuff. Will take this branch for a spin.

from astroid.

Just as if we needed even more incentives...
This just in:

---

WebKitGTK+ Security Advisory WSA-2017-0008

Date reported : October 18, 2017
Advisory ID : WSA-2017-0008
Advisory URL : https://webkitgtk.org/security/WSA-2017-0008.html
CVE identifiers : CVE-2017-7081, CVE-2017-7087, CVE-2017-7089,
CVE-2017-7090, CVE-2017-7091, CVE-2017-7092,
CVE-2017-7093, CVE-2017-7094, CVE-2017-7095,
CVE-2017-7096, CVE-2017-7098, CVE-2017-7099,
CVE-2017-7100, CVE-2017-7102, CVE-2017-7104,
CVE-2017-7107, CVE-2017-7109, CVE-2017-7111,
CVE-2017-7117, CVE-2017-7120, CVE-2017-7142.

Several vulnerabilities were discovered in WebKitGTK+.
[...]

from astroid.

ff2000 writes on January 10, 2016 8:36:

Do you plan to port astroid to WebKit2?
I could try it myself, but I have no idea about both WebKit1 and WebKit2 Apis Do you have some hints where to start?

I approve.

I actually tried this first, and I believe you can find my attempts in
the early git-history. But webkit2 is more separated and as far as I
could see we would not be able to access the DOM-tree as we do now, but
rather as a plugin.

It might actually work better to have some more separation between
astroid and the UI, as long as it can still be manipulated
programatically.

regards, gaute

from astroid.

webkitgtk-2.10 comes with some webkitdom-headers, which look as if they allow modification of the dom-tree - isn't that enough?

Also I think I will wait with porting until 0.5 is released. I have never ever done GTK development (besides one hello world window some time ago), so I need some small issues for practising first ;)

from astroid.

ff2000 writes on January 18, 2016 10:51:

webkitgtk-2.10 comes with some webkitdom-headers, which look as if they allow modification of the dom-tree - isn't that enough?

Hopefully! Let's do some tests.

Also I think I will wait with porting until 0.5 is released. I have never ever done GTK development (besides one hello world window some time ago), so I need some small issues for practising first ;)

#63 hint hint

from astroid.

Excerpts from Gaute Hope's message of Januar 18, 2016 10:54 :

ff2000 writes on January 18, 2016 10:51:

webkitgtk-2.10 comes with some webkitdom-headers, which look as if they allow modification of the dom-tree - isn't that enough?

Hopefully! Let's do some tests.

Also I think I will wait with porting until 0.5 is released. I have never ever done GTK development (besides one hello world window some time ago), so I need some small issues for practising first ;)

#63 hint hint

Yay, that's what I planned ;)

from astroid.

@emdete also mentioned https://github.com/litehtml

from astroid.

Michael Turquette writes on juni 24, 2016 6:52:

Upvoting this issue: 👍

Homebrew on macOS only supports newer webkitgtk:

$ brew info webkitgtk
webkitgtk: stable 2.10.9 (bottled)
Full-featured Gtk+ port of the WebKit rendering engine
http://webkitgtk.org
/usr/local/Cellar/webkitgtk/2.10.9_1 (331 files, 40.2M) *
  Poured from bottle on 2016-06-22 at 12:36:40
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/webkitgtk.rb
==> Dependencies
Build: xz ✔, cmake ✔
Required: gtk+3 ✔, libsoup ✔, enchant ✔, webp ✔

Two questions:

1. What version of webkitgtk does this translate to in GNU Linux/Debian packaging speak? I'm always confused by the package naming conventions... (see this stackoverflow thread)

I tried to get this running using homebrew at some point. I think the
last version of webkitgtk that works with webkit1 is 2.4.11.

1. Should the method for version checking webkitgtk in scons be updated to reflect the library name in macOS? See:

$ scons --prefix=/usr/local build
scons: Reading SConscript files ...
building version v0.5-228-g31ad8c35 (git)..
debug flag enabled: True
Checking for pkg-config... yes
Checking for gtkmm-3.0 >= 3.10... yes
Checking for glibmm-2.4... yes
Checking for gmime-2.6 >= 2.6.18... yes
Checking for webkitgtk-3.0... no
webkitgtk not found.

No, because currently astroid needs webkitgtk, not webkit2gtk. So the
check fails because brew-webkit is webkit2gtk (in the linux world, Arch
at least).

The main stopper when I initially tried to use webkit2 was the DOM
access methods, I think this is easier now. A lot of the other API is
very similar.

Regards, Gaute

from astroid.

another idea was to look at http://netsurf-browser.org/. out of security reasons i would not like to base on such complex systems as webkit or gecko to display simple things like emails. still others lack important features like JS or domtreechanges. but the guys are working on it. beside litehtml the netsurf seems to be a quite cool project. so instead investing into migration to webkit2 i would suggest to take a look into simple html render libraries.

from astroid.

Any chance the update to webkit2 will happen soon?

webkit 1 (aka the webkitgtk3 dependency in this project) is no longer maintained and has had over 200 security vulnerabilities. source, see last section

Distributions will likely be removing it soon.

This week I found astroid mail, and made a pretty awesome isync/notmuch/msmtp/astroid setup. I also created a flatpak package for astroid (see #224), which is where I found out this project is using such an old an insecure webkit version.

As a result of this, I just can't use astroid yet, nor do I want to release my flatpak package until this fixed. (the flatpak package takes over 9hrs to compile because of webkit1).

astroid is a wicked cool MUA though. I wish I had time to help update it.

from astroid.

I did some googleing, and I know nothing about the webkitgtk api. But from the resources below it looks like you can grab and manipulate the dom in webkit2.

• https://blogs.igalia.com/carlosgc/2013/09/10/webkit2gtk-web-process-extensions/
• https://webkitgtk.org/reference/webkit2gtk/stable/WebKitWebPage.html#webkit-web-page-get-dom-document
• https://webkitgtk.org/reference/webkitdomgtk/stable/WebKitDOMDocument.html

from astroid.

from astroid.

Web extensions are written in Javascript, and sadly these days im more comfortable with JS than C.

I don't have something much extra time, but if it's straightforward I’d offer to help write the web extension bit. Where's the code that needs to be ported?

from astroid.

from astroid.

Do you have any docs or examples I could have a look at on how that would work?

from astroid.

Oops, there's been a misunderstanding on my part. I understood web extensions to mean the new browser extension API used by Chrome and Firefox.

But "web extensions" in the context of webkitgtk are something else specific to gtk and are definitely written in C, though they do support injecting JS into the browser context.

So it doesn't look like I'll be helpful in this case :\

from astroid.

from astroid.

from astroid.

from astroid.

Interesting!

Can you make a small writeup on what needs to be done in JS exactly? I'm not familiar with the codebase at all, but if you could provide a list of things that need to be done I could tackle those.

from astroid.

from astroid.

Now loading javascript.

from astroid.

There's some test code that runs some javascript code defined in the lib modifying the document: https://github.com/gauteh/astroid/blob/webkit2js/src/modes/thread_view/thread_view.cc#L629

from astroid.

#408

from astroid.

I haven't had a chance to try and build on macOS yet. Has anyone else?

from astroid.

@mturquette: Check out: https://github.com/astroidmail/homebrew-astroid for a version that works with astroid master and webkit1. I think @yeled might have tried to build using webkit2.

from astroid.

Yeah, I doubt we are affected unless you view a HTML part - but definitely time to get to webkit2 as soon as possible.

from astroid.

Downstream:

• https://bugzilla.redhat.com/show_bug.cgi?id=1438659

from astroid.

Please see #455 for currently active solution to porting to webkit2. This uses C/C++ only by creating an webextension. Google Protobuf used for communication through unix domain sockets.

from astroid.