Comments (7)
thp
commented on July 29, 2024
The filter mechanism is designed to cut out parts of pages that always change. Do you have an example URL for which this happens? It might make sense to cache the last two versions, but then it should probably be per-URL and also configurable (e.g. it might make sense to cache the last 7 versions for a page that rotates it content on a daily basis, with repeating content every week).
from urlwatch.
protist
commented on July 29, 2024
Yes, I am already using the filter mechanism for a few pages, which works brilliantly, by the way. However, these changes appear to occur for almost the whole of this page. (In this case, I should probably just watch the mercurial project instead.)
I have urlwatch operating hourly by cron. This page has been reported as changing to nonsense five times in the last six days, then reverting back on the next watch, or sometimes on the same watch (not sure how that works). I've reproduced the last nonsense conversion below.
To be honest, this is really just a bug in the watched page, so I understand if it's not a priority to address.
***************************************************************************
CHANGED: http://sjl.bitbucket.org/gundo.vim/
***************************************************************************
--- @ Wed, 19 Aug 2015 07:00:04 +1000
+++ @ Wed, 19 Aug 2015 08:00:04 +1000
@@ -1,544 +1,95 @@
-<!DOCTYPE html>
-<html>
-
- <title>Gundo - Visualize your Vim Undo Tree</title>
- <base href="/gundo.vim/">
-
- <link rel="stylesheet/less" href="style.less" type="text/css">
- <script src="less.js" type="text/javascript"></script>
-
- <script type="text/javascript">
- /* <![CDATA[ */
- (function() {
- var s = document.createElement('script'), t = document.getElementsByTagName('script')[0];
-
- s.type = 'text/javascript';
- s.async = true;
- s.src = 'http://api.flattr.com/js/0.5.0/load.js?mode=auto';
-
- t.parentNode.insertBefore(s, t);
- })();
- /* ]]> */
- </script>
-
- <script type="text/javascript">
- var _gaq = _gaq || [];
- _gaq.push(['_setAccount', 'UA-15328874-3']);
- _gaq.push(['_trackPageview']);
-
- (function() {
- var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
- ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
- var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
- })();
- </script>
- </head>
-
- <body>
- <div id="repos">
- <a href="http://bitbucket.org/sjl/gundo.vim/">Mercurial Repository</a><br/>
- <a href="http://github.com/sjl/gundo.vim/">Git Repository</a><br/>
-
- <a class="FlattrButton" style="display:none;" rev="flattr;button:compact;" href="http://bitbucket.org/sjl/gundo.vim/"></a>
- </div>
- <div class="wrap">
- <header>
- <h1>Gundo</h1>
- <h2>Graph your Vim undo tree in style.</h2>
- </header>
- <section>
- <a href="http://www.flickr.com/photos/sjl7678/5093114605/"
- title="gundo by stevelosh, on Flickr"
- ><img src="http://farm5.static.flickr.com/4113/5093114605_ebc46d6494_m.jpg"
- width="234" height="240" alt="gundo" /></a>
-
-
- <p>
- You know that Vim lets you undo changes like any text editor. What you might
- not know is that it doesn't just keep a list of your changes — it keeps
- a <a href="http://vim.wikia.com/wiki/Using_undo_branches">tree</a> of them.
- </p>
-
- <p>
- Say you make a change (call it X), undo that change, and then make another
- change (call it Y). With most editors, change X is now gone forever. With Vim
- you can get it back.
- </p>
-
- <p>
- The problem is that trying to do this in the real world is painful. Vim gives
- you an <code>:undolist</code> command that shows you the leaves of the tree. Good luck
- finding the change you want in that list.
- </p>
-
- <p>
- Gundo is a plugin to make browsing this ridiculously powerful undo tree less
- painful.
- </p>
-
- <p>
- Check out this
- <a href="http://screenr.com/M9l">quick screencast</a>
- if you want to see it in action.
- </p>
- </section>
- <nav>
- <ol>
- <li><a href="#requirements">Requirements</a></li>
- <li><a href="#installation">Installation</a></li>
- <li><a href="#usage">Usage</a></li>
- <li>
- <a href="#configuration">Configuration</a>
- <ul>
- <li><a href="#gundo_width">g:gundo_width</a></li>
- <li><a href="#gundo_preview_height">g:gundo_preview_height</a></li>
- <li><a href="#gundo_preview_bottom">g:gundo_preview_bottom</a></li>
- <li><a href="#gundo_right">g:gundo_right</a></li>
- <li><a href="#gundo_help">g:gundo_help</a></li>
- <li><a href="#gundo_disable">g:gundo_disable</a></li>
- <li><a href="#gundo_map_move">g:gundo_map_move_[older/newer]</a></li>
- <li><a href="#gundo_close_on_revert">g:gundo_close_on_revert</a></li>
- <li><a href="#gundo_statusline">g:gundo_[preview/tree]_statusline</a></li>
- <li><a href="#gundo_auto_preview">g:gundo_auto_preview</a></li>
- <li><a href="#gundo_playback_delay">g:gundo_playback_delay</a></li>
- </ul>
- </li>
- <li><a href="#license">License</a></li>
- <li><a href="#bugs">Bugs</a></li>
- <li><a href="#contributing">Contributing</a></li>
- <li><a href="#changelog">Changelog</a></li>
- <li><a href="#credits">Credits</a></li>
- </ol>
- </nav>
- <section>
- <a name="requirements"></a>
- <h1>Requirements</h1>
-
- <p>
- Gundo requires Vim 7.3+ compiled with Python support, and Python 2.4+.
- </p>
- </section>
- <section>
- <a name="installation"></a>
- <h1>Installation</h1>
-
- <p>Use Pathogen. Don't use pathogen? Start.</p>
- <pre>hg clone http://bitbucket.org/sjl/gundo.vim ~/.vim/bundle/gundo</pre>
-
- <p>There's a git mirror if you prefer:</p>
- <pre>git clone http://github.com/sjl/gundo.vim.git ~/.vim/bundle/gundo</pre>
-
- <p>Add a mapping to your ~/.vimrc (change the key to suit your taste):</p>
- <pre>nnoremap <F5> :GundoToggle<CR></pre>
- </section>
- <section>
- <a name="usage"></a>
- <h1>Usage</h1>
-
- <p>
- We'll get to the technical details later, but if you're a human the first
- thing you need to do is add a mapping to your vimrc file to toggle the undo
- graph:
- </p>
-
- <pre>nnoremap <F5> :GundoToggle<CR></pre>
-
- <p>
- Change the mapped key to suit your taste. We'll stick with <code><F5></code> because that's
- what the author uses.
- </p>
-
- <p>
- Now you can press <code><F5></code> to toggle the undo graph and preview pane, which will
- look something like this:
- </p>
-
-<pre>
- Undo graph File
-+-----------------------------------+---------------------------+
-| " Gundo for something.txt [1] |one |
-| " j/k - move between undo states |two |
-| " <cr> - revert to that state |three |
-| |five |
-| @ [5] 3 hours ago | |
-| | | |
-| | o [4] 4 hours ago | |
-| | | | |
-| o | [3] 4 hours ago | |
-| | | | |
-| o | [2] 4 hours ago | |
-| |/ | |
-| o [1] 4 hours ago | |
-| | | |
-| o [0] Original | |
-+-----------------------------------+ |
-| --- 3 2010-10-12 06:27:35 PM | |
-| +++ 5 2010-10-12 07:38:37 PM | |
-| @@ -1,3 +1,4 | |
-| one | |
-| two | |
-| three | |
-| +five | |
-+-----------------------------------+---------------------------+
- Preview pane
-</pre>
-
- <p>
- Your current position in the undo tree is marked with an <code>@</code> character. Other
- nodes are marked with an <code>o</code> character.
- </p>
-
- <p>
- When you toggle open the graph Gundo will put your cursor on your current
- position in the tree. You can move up and down the graph with the <code>j</code> and
- <code>k</code> keys.
- </p>
-
- <p>
- You can move to the top of the graph (the newest state) with <code>gg</code> and to the
- bottom of the graph (the oldest state) with <code>G</code>.
- </p>
-
- <p>
- As you move between undo states the preview pane will show you a unified diff
- of the change that state made.
- </p>
-
- <p>
- Pressing <code>return</code> on a state (or double clicking on it) will
- revert the contents of the file to match that state.
- </p>
-
- <p>
- You can use <code>p</code> on a state to make the preview window show the
- diff between your current state and the selected state, instead of a preview
- of what the selected state changed.
- </p>
-
- <p>
- Pressing <code>P</code> while on a state will initiate "play to" mode targeted at that
- state. This will replay all the changes between your current state and the
- target, with a slight pause after each change. It's mostly useless, but can be
- fun to watch and see where your editing lags — that might be a good place to
- define a new mapping to speed up your editing.
- </p>
-
- <p>
- Pressing <code>q</code> while in the undo graph will close it. You can also just press your
- toggle mapping key.
- </p>
- </section>
- <section>
- <a name="configuration"></a>
- <h1>Configuration</h1>
-
- <p>
- You can tweak the behavior of Gundo by setting a few variables
- in your :vimrc file. For example:
- </p>
-
-<pre>
-let g:gundo_width = 60
-let g:gundo_preview_height = 40
-let g:gundo_right = 1
-</pre>
-
- <a name="gundo_width"></a>
- <h2>g:gundo_width</h2>
-
- <p>Set the horizontal width of the Gundo graph (and preview).</p>
-
- <p>Default: 45</p>
-
- <a name="gundo_preview_height"></a>
- <h2>g:gundo_preview_height</h2>
-
- <p>Set the vertical height of the Gundo preview.</p>
-
- <p>Default: 15</p>
-
- <a name="gundo_preview_bottom"></a>
- <h2>g:gundo_preview_bottom</h2>
-
- <p>
- Force the preview window below current windows instead of below
- the graph. This gives the preview window more space to show the
- unified diff.
- </p>
-
- <p>Example:</p>
-
-<pre>
-+--------+ +--------+
-!g! ! ! !g!
-!g! ! or ! !g!
-!g!______! !______!g!
-!g!pppppp! !pppppp!g!
-+--------+ +--------+
-</pre>
-
- <p>Default: 0</p>
-
- <a name="gundo_right"></a>
- <h2>g:gundo_right</h2>
-
- <p>
- Set this to 1 to make the Gundo graph (and preview) open on the
- right side instead of the left.
- </p>
-
- <p>Default: 0 (off, open on the left side)</p>
-
- <a name="gundo_help"></a>
- <h2>g:gundo_help</h2>
-
- <p>Set this to 0 to disable the help text in the Gundo graph window.</p>
-
- <p>Default: 1 (show the help)</p>
-
- <a name="gundo_disable"></a>
- <h2>g:gundo_disable</h2>
-
- <p>Set this to 1 to disable Gundo entirely.</p>
-
- <p>
- Useful if you use the same <code>~/.vim</code> folder on
- multiple machines, and some of them may not have Python support.
- </p>
-
- <p>Default: 0 (Gundo is enabled as usual)</p>
-
- <a name="gundo_map_move"></a>
- <h2>g:gundo_map_move_[older/newer]</h2>
-
- <p>
- These options let you change the keys that navigate the
- undo graph. This is useful if you use a Dvorak keyboard and
- have changed your movement keys.
- </p>
-
- <p>
- Default:<br/>
- gundo_map_move_older: "j"<br/>
- gundo_map_move_newer: "k"
- </p>
-
- <a name="gundo_close_on_revert"></a>
- <h2>g:gundo_close_on_revert</h2>
-
- <p>Set this to 1 to automatically close the Gundo windows when reverting.</p>
-
- <p>Default: 0 (windows do not automatically close)</p>
-
- <a name="gundo_statusline"></a>
- <h2>g:gundo_[preview/tree]_statusline</h2>
-
- <p>
- Set this to 0 to disable automatically rendering preview diffs as you move
- through the undo tree (you can still render a specific diff with r). This can
- be useful on large files and undo trees to speed up Gundo.
- </p>
-
- <p>Default: unset (windows use the default statusline)</p>
-
-
- <a name="#gundo_auto_preview"></a>
- <h2>g:gundo_auto_preview</h2>
-
- <p>Set this to 1 to rendering diff automatically with cursor move.</p>
-
- <p>Default: 1 (auto preview diff)</p>
-
- <a name="#gundo_playback_delay"></a>
- <h2>g:gundo_playback_delay</h2>
-
- <p>
- This is the delay in milliseconds between each change when running 'play to'
- mode. Set this to a higher number for a slower playback or to a lower number
- for a faster playback.
- </p>
-
- <p>Default: 60</p>
- </section>
- <section>
- <a name="license"></a>
- <h1>License</h1>
-
- <p><a href="http://www.opensource.org/licenses/gpl-2.0.php">GPLv2+</a>.</p>
- </section>
- <section>
- <a name="bugs"></a>
- <h1>Bugs</h1>
-
- <p>
- If you find a bug please post it on the
- <a href="http://bitbucket.org/sjl/gundo.vim/issues?status=new&status=open">issue tracker</a>.
- </p>
- </section>
- <section>
- <a name="contributing"></a>
- <h1>Contributing</h1>
-
- <p>
- Fork the repository on
- <a href="http://bitbucket.org/sjl/gundo.vim/">BitBucket</a>
- or
- <a href="http://github.com/sjl/gundo.vim/">GitHub</a>
- and send a pull request.
- </p>
-
- <p>Make sure you document your changes in the following places:</p>
-
- <ul>
- <li>The <code>README.markdown</code> file.</li>
- <li>The <code>site/index.html</code> file.</li>
- <li>The <code>doc/gundo.txt</code> file.</li>
- </ul>
- </section>
- <section>
- <a name="changelog"></a>
- <h1>Changelog</h1>
-
- <ol class="changelog">
- <li>v2.5.0
- <ul>
- <li>
- Fix the help window to take custom mappings into account.
- </li>
- <li>
- Add <code>g:gundo_playback_delay</code> option.
- </li>
- </ul>
- </li>
- <li>v2.4.0
- <ul>
- <li>
- Add auto preview option.
- </li>
- <li>
- Add 'r' mapping to preview current state.
- </li>
- <li>
- Add public <code>gundo#GundoShow()</code> and <code>gundo#GundoHide()</code> functions.
- </li>
- </ul>
- </li>
- <li>v2.3.0
- <ul>
- <li>
- Add statusline configuration.
- </li>
- </ul>
- </li>
- <li>v2.2.2
- <ul>
- <li>
- More performance improvements.
- </li>
- </ul>
- </li>
- <li>v2.2.1
- <ul>
- <li>
- Refactoring and performance improvements.
- </li>
- </ul>
- </li>
- <li>v2.2.0
- <ul>
- <li>
- Add the <code>g:gundo_close_on_revert</code> setting.
- </li>
- <li>
- Fix a bug with the <code>splitbelow</code> setting.
- </li>
- </ul>
- </li>
- <li>v2.1.1
- <ul>
- <li>
- Fix a bug with the movement key mappings.
- </li>
- </ul>
- </li>
- <li>v2.1.0
- <ul>
- <li>
- Warnings about having an incompatible Vim and/or
- Python installation are now deferred until the
- first time you try to use Gundo, instead of being
- displayed on launch.
- </li>
- <li>
- The <code>j</code> and <code>k</code> mappings are
- now configurable with
- <code>g:gundo_map_move_older</code> and
- <code>g:gundo_map_move_newer</code>.
- </li>
- <li>
- The <code>o</code>, <code>Up</code> and
- <code>Down</code> keys are now mapped in the
- Gundo pane.
- </li>
- <li>
- Improve and add several unit tests for Gundo.
- </li>
- </ul>
- </li>
- <li>v2.0.0
- <ul>
- <li>
- Make <code>GundoToggle</code> close the Gundo windows if they're
- visible but not the current window, instead of moving to them.
- </li>
- <li>Add the <code>g:gundo_disable</code> setting.</li>
- <li>Add the <code>g:gundo_help</code> setting.</li>
- <li>
- Add the <code>p</code> mapping to preview the result of
- reverting to the selected state.
- </li>
- <li>Fix movement commands with counts in the graph.</li>
- </ul>
- </li>
- <li>v1.0.0
- <ul>
- <li>Initial stable release.</li>
- </ul>
- </li>
- </ol>
- </section>
- <section>
- <a name="credits"></a>
- <h1>Credits</h1>
-
- <p>
- The graphing code was all taken from Mercurial, hence the
- GPLv2+ license.
- </p>
-
- <p>
- The plugin was heavily inspired by histwin.vim, and the code
- for scratch.vim helped the author get started.
- </p>
- </section>
- <footer>
- Gundo was written by
- <a href="http://stevelosh.com">Steve Losh</a>
- with a lot of help from others.
- </footer>
- </div>
- <script type="text/javascript">
- var _gauges = _gauges || [];
- (function() {
- var t = document.createElement('script');
- t.type = 'text/javascript';
- t.async = true;
- t.id = 'gauges-tracker';
- t.setAttribute('data-site-id', '4f843f8c613f5d65280000e6');
- t.src = '//secure.gaug.es/track.js';
- var s = document.getElementsByTagName('script')[0];
- s.parentNode.insertBefore(t, s);
- })();
- </script>
- </body>
-</html>
+��
+�a0�L��>9'$۶-y$9�^:�dvR%>Z޹գ�>pFdˎp3R~�w'4u꺩}8�~tzeC>�Wfk�}��E �v/M~p��{z7�v(a9V�͂Xu��QH=�{
+\��u��7/�u{M8M:�WW)��zx܈Zgi(U�vY�baQ+/+S᮰��v�\4p�in{ۃ(n腯oG;aka:ڎVVQtrոxsd'"H
+
+W�py~"]ZJw�|b*M
+{oa`k�}5>fxGC5ܗo�/;^:�>�<��/_z|
+6#ByR CV�-h��qHe]JV�5DqUDX>
++�
+`P�(d9 k�mR�Q N�H�B_RiTY_'p*,�t^K{�b/>x��ʲ4nd�̥BJh�Ǝ�+ۓӑ�JT4)BBIdS-LSA�րkֹz%FɎ�f+Xo^73ur�PR<5k��l<.I<)Q1<�7�=<ΔRa-Gcvh��DB;�b}i�;կ�~P�ShA5\�Ad4<US/E8�vWb_fw*Ei-�0_�lh(= ~�}Ue#sW]h&�^]z>6n1ZU1J۫lDMϧ6N�tNFEٚT`��_[Հѫ/�Iok!
+�'i_�L�[jn7�VraF !?n<M�j^|&� ��'ʩ�pmvmA��v�=��P}hqG~Y�::� <�!<h_Ha ~�h�_٩� �o�yN
+T[x49~*S��\qc:Pə
+l�`t�0^�E��̳1�3��0�B(9g8HJ,��96)>n��cj칓_�՞CuO'I�*�ؘQ7@@4X�E{}���e0ɰ92MG^P��â�T
+pK�`%���_}�LT?�Cd1 ��
+KP���!
+
+�Ml6o Z�q5�[%i2]f��IE2iSJ3HQz#�ۺ�t��d`WlbvI{�)<��hW�H�8%zl#+*
+~)�,#M�RKK.su9q
+rYZ̡�U�{d@-N_ra@/k"h
+�rl��H�TAJ/�N�UDѪ5Cv6$:F'��ȱJ�4�CW�)CM-nв1H3<Ʊ%2MH�-_m@\k [<mÏ +̧hV3\.é�¬4"�6+]Й�:l��"?�h0�Kmp�G
+�$(//?����̧.w<�'r�V\eV5lvn�7bnr3�hs0?�VWc*gIy�&aW1o[VȅR`^�h K�
+�[ڥ�ܑ|;6li��cT;╌Xgצ�`�CF:�ۧ5;kkuG7
+lU!�(! κ
+x�"jJ�
+r�8Q>4V='(\#
+�'a�%You�d
+{pB֗ĽB?'h0�@I3lT~��"4���W/0w$[o\^�fov8Zԑ%i;/F�8Vt��mmIo�IʸŨ+MZ85��Ԛ)�3`f%
+Mw�KQC)Cݑ Ъ�]Dc+Dd���Ԩ{�@#GƵ֓^ �T�ч*3���:g*e(ulN����ax'n�ȊS+q
+r�}Q*3P?3D
+J!==n
+)Wx\�0`
+b�-)$g<JQ ܑQb1�18�TR#M��)y�%.`F�my
+w�Κ6�n:[Utk@!bfmn�W�F̂u�)~κ3���=5xoAѼ~Cqlz���?'G㤿�|<?�>y3zp�?
+?0{3~fl˧ƣo{gO鮿埇oyfw�v|:<xi��,_*É`�M7��<|rQo�?y'wAdó�Þ¾yvç»·��O�f/{xGo�t *zÊŒTL~x'���QQr*#_xÞŽ�;iS8|�?F9G9'.'8
+'cvtÛ¼ï‰�1 kC�L_&O�8]󱂃5|&(!xÎŒ9n>_&˶4ÉŠÍ‹'�M./LP쑵[�gÑ£{Y�Ri:�'|J 8]�svrsbJ%kXv�KN?D�'<w8ry{aYIO��LU=�H9��hpH�í ”�m
+%Y[om��NWtY�a$mZ|$v<U�l�
+�t�h�9!>l���b�D2��sA�*O9�ىm'\O�(q^3����T�.�$
+B�j��q���n6M�%ZH�Fj^0�q0v:B!a-o�Y�"�o*cՒ1h�j�$3A�W�@W9���.tFG3AE
+�*G+�ς�M��J�j�`aYR&�V9#!ŻF]0LHj,%aՄ�Y` ZzE4sb;q3z���IhN^'�fIb\�fhr�%"lѕ2T<l�Fx�Bo�E"�!Ht|�rEaCT=�EzumV~t\G�>RįP,S_R]�_N�h:'*=kr��\gQ
+�n�ps=u-�hÙª[#\ׂ9�5l�4����Í u]���QG�[~$�+fw��SlWoY;w6:_ׯYYYqF@���Tm:+88-5 1Qf�GF��Pl2v#\#n3Â�F
+T��[R/�$�Y,r�6" 3ӦbŰ}o֔4
+Bu> D��]``�k3mLS0V�_ ֌"La}"��%A�"w�\: �e#�xL�%xwtq^y/(*m}FʉtSیj?kViMhjs��>8ѦzONkfў��$]!}u`bhF*~jF]I1�H�%"Tr
+����{Ӊ8[d&H?ىN4�A�tPo����ℿ��.8%jbk��$1o1d=$�&uݎ9�zGWa3 FI��qk�5bp��DQf�y
+Ga��
+A�M0�y#C
+]& $(p^y�� �?MI:fT%?:��-G���g;�<�LC|9zv�~ϧ��ߊLZFB3/-�`B
+u ,Bk2e�ZĢA*�5_9#S�I�KƲ�o>
+f�٣Pݣ�T��j%M�&
+uu6G#w\�GB'Þ¬4�r O
+p/ E�;
+Pl; �{dܦ܊gh_,cfdm8�n[�i~uc�kJ{v�k�Rgp�h�+�EEy\>��uA�x�jхN^2_ MwW�#�S���V+�#^1N-sYҞ5�K�O]#�: &'�`=PDj˩J" x4u�z84H�N �<T-eڀf�~�i�?|'='K�
+7��H��'o8jq��]�=~qbJỷA�lzq|m՚Xe?mCc`)*n1bKE
+`)CD1+z�d�a!p�X!9��W>&uCq޺ �p�Y,|sdJIk=y)0Fτ/�Q
+Vc
+UҀ��fI-v+�G[v�I)o&
+nC�{9^D@Ú°n
+LݵV/%20v1SdF���1!Ԁӵ�M~G4-ob_xSE jqa
+lG&<EϚeQK(;u踕x~`ò:BFo[x=� '�[QN�,�pZ����̢.\QYXVh
+ =�i�(qQc[\�~̸�W'<ͤ+,+yH�B
+� �ո$V��z/)@
+b �R�R3S+r+
+P3�&*Tt�GDr&�V!��^��(.�)�q+iB'�|�d8�7�ka=�ìVy\p'�Ɉ�hwF�u1�iB��0�����D�eU?�vY&�MݲQ}ϒt]��'#qF~
+��<v�u9ZG$�]+;ce�?msۊfI%˦?2�@,!P#�R<cq,f
+}[sVB_�Z&Er6a�y|[xo'�)[x#����߱�7=�M[YJ�Ydq��6�UW>̹5xw�ko�x?uorHYy)^fIU�Twd�/\|}+Hր��ŬSWtS~%S�?Vi{gM�)%�v��P2
***************************************************************************
--
urlwatch 1.15, Copyright 2008-2011 Thomas Perl
Website: http://thp.io/2008/urlwatch/
watched 23 URLs in 5 seconds
from urlwatch.
thp
commented on July 29, 2024
Maybe some feature that keeps the last N versions of the page and when diffing checks if one of them matches could do the trick here? As a per-URL override, that is.
from urlwatch.
protist
commented on July 29, 2024
Yes, that might work. One part that confuses me would be what to do about the "false positive" page. Consider the following.
- urlwatch checks the page, which is constant in its steady state.
- The page changes temporarily to the nonsense page. urlwatch doesn't report immediately on the change. The previous page is still cached. Presumably this nonsense page is also cached?
- The page reverts to the steady state. urlwatch checks the original cache, sees no difference and does not report.
However, what happens now with the nonsense page cache? If, in the future, the page changes to the nonsense page permanently, urlwatch should notify. Hence, I guess the nonsense page's cache should be destroyed when the page reverts?
Another thing… perhaps users could specify how many unique versions to keep, and a maximum age (measured chronologically or "checks ago") of cached pages. If the nonsense page persists for up to n hours, or m checking cycles, urlwatch can still ignore it.
from urlwatch.
thp
commented on July 29, 2024
I would assume that urlwatch would report the first change but would not report the change back, and any subsequent change will then get reported again?
So assuming we have these states: A, B, C, D (and considering "B" the intermediate trash state)
These state changes: A->B->A->B->C->B->D
Then it would report A->B, skip B->A (A is already known), skip A->B (B is already known), report B->C, skip C->B (B is already known), report B->D
This is not ideal, as we want to report the B-to-something changes with the state before B (assuming B contains garbage), so with A->B->C we want to report a diff of A->C when the actual change is B->C, right? Would a filter that can mark a result as "garbage" work in that way? That we just filter garbage? I do have a page where it actually sometimes prints out "No connection to the database" instead of a list of items, so that would solve that problem as well (filter would then be "If the text contains No connection to the database, ignore for now").
from urlwatch.
protist
commented on July 29, 2024
Yes, this makes much sense overall, in both the reporting and diff-ing.
However, if the "junk" results in different pages each time, which (I expect) is occurring in my example above, then this would not solve that problem. The only solution I can think of is to only report when the junk page is present after multiple checks.
I appreciate that this may be confusing to the end user, and will result in a delay in reporting, so even though it'd solve my problem in theory, I'm not sure that it's ideal.
from urlwatch.
thp
commented on July 29, 2024
The new SQLite3/minidb-based storage method in urlwatch 2 does keep old versions around until manually purged, so it might be possible to implement something like this now by looking at the last N versions of a page.
from urlwatch.
Related Issues (20)
- Reporting blanks HOT 28
- add support to specify multiple recipients per URL HOT 7
- YAML Anchors/Aliases not working HOT 4
- CSS Filter "AttributeError: 'CSSSelector' object has no attribute 'evaluate'" HOT 2
- FEATURE: Support multiple reporters with different options HOT 6
- Meaning of max_tries is confusing
- urlwatch stopped working HOT 4
- sendmail is not documented HOT 2
- Randomly "not enough values to unpack" Python errors HOT 4
- Cache inconsistency creating new items from nowhere HOT 3
- Feature request: Extension of regex filtering to extract data HOT 7
- Consider releasing version 2.29 HOT 5
- Question - Report http errors only once HOT 5
- urlwatch 2.25-1 on Debian Stable 12.5 (navigate fails) HOT 3
- Enable/disable job from the command line HOT 1
- urlwatch moans when supplying --config HOT 1
- urlwatch 2.25 - AttributeError: 'list' object has no attribute 'read' HOT 2
- XML parsing with CDATA not working HOT 6
- urlwatch 2.28: html: separate: true not sending separate emails HOT 2
- --test-filter works but not with a normal execution HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from urlwatch.