September 2009 - netsurf-commits - listmaster.pepperfish.net

r9561 jmb - /trunk/netsurf/Docs/ideas/cache.txt

by netsurf＠semichrome.net

Author: jmb Date: Tue Sep 8 14:24:11 2009 New Revision: 9561 URL: http://source.netsurf-browser.org?rev=9561&view=rev Log: Musings on caching Added: trunk/netsurf/Docs/ideas/cache.txt Added: trunk/netsurf/Docs/ideas/cache.txt URL: http://source.netsurf-browser.org/trunk/netsurf/Docs/ideas/cache.txt?rev=... ============================================================================== --- trunk/netsurf/Docs/ideas/cache.txt (added) +++ trunk/netsurf/Docs/ideas/cache.txt Tue Sep 8 14:24:11 2009 @@ -1,0 +1,168 @@ +Content caching +=============== + +NetSurf's existing fetch/cache architecture has a number of problems: + +1) Content dependencies are not modelled. +2) Content source data for non-shareable contents is duplicated. +3) Detection of content sharability is dependent on Content-Type, which + requires content cloning (which will fail for dependent contents). +4) Detection of cycles in content dependency graphs is not performed + (e.g. content1 includes content2, which includes content1). +5) All content caching is in-memory, there's no offline storage. + +Proposal +-------- + +A split-level cache. + +Low-level cache: + + + Responsible for source data (+header) management. + + Interfaces with low-level fetch system to retrieve data from network. + + Is responsible for offline storage (if any) of cache objects. + + Returns opaque handles to low-level cache objects. + + Handles HTTP redirects, recording URLs encountered when retrieving resource. + + May perform content-type sniffing (requires usage context) + +High-level cache: + + + Responsible for content objects. + + Tracks content dependencies (and potential cycles). + + Returns opaque handles to content objects. + + Manages content sharability. + + Contents with unknown types are never shared and thus get unique handles. + + Content handles <> content objects: they're an indirection mechanism. + +Example: retrieving a top-level resource +---------------------------------------- + + 1) Client requests an URL, specifying no parent handle. + 2) High-level cache asks low-level cache for low-level handle for URL. + 3) Low-level cache looks for appropriate object in its index. + a) it finds one that's not stale and returns its handle + b) it finds only stale entries, or no appropiate entry, + so allocates a new entry, requests a fetch for it, + and returns the handle. + 4) High-level cache looks for content objects that are using the low-level + handle. + a) it finds one that's shareable and selects its handle for use. + b) it finds only non-shareable entries, or no appropriate entry, + so allocates a new entry and selects its handle for use. + 5) High-level cache registers the parent and client with the selected handle, + then returns the selected handle. + 6) Client carries on, happy in the knowledge that a content is available. + +Example: retrieving a child resource +------------------------------------ + + 1) Client requests an URL, specifying parent handle. + 2) High-level cache searches parent+ancestors for requested URL. + a) it finds the URL, so returns a non-fatal error. + b) it does not find the URL, so proceeds from step 2 of the + top-level resource algorithm. + + NOTE: this approach means that shareable contents may have multiple parents. + +Handling of contents of unknown type +------------------------------------ + + Contents of unknown type are, by definition, not shareable. Therefore, each + client will be issued with a different content handle. + + Content types are only known once a resource's headers are fetched (or once + the type has been sniffed from the resource's data when the headers are + inconclusive). + + As a resource is fetched, users of the resource are informed of the fetch + status. Therefore, the high-level cache is always informed of fetch progress. + Cache clients need not care about this: they are simply interested in + a content's readiness for use. + + When the high-level cache is informed of a low-level cache object's type, + it is in a position to determine whether the corresponding content handles + can share a single content object or not. + + If it detects that a single content object may be shared by multiple handles, + it simply creates the content object and registers each of the handles as + a user of the content. + + If it detects that each handle requires a separate content object, then it + will create a content object for each handle and register the handle as a + user. + + This approach requires that clients of the high-level cache get issued with + handles to content objects, rather than content objects (so that the decision + whether to create multiple content objects can be deferred until suitable + information is available). + + Handles with no associated content object will act as if they had a content + object that was not ready for use. + +A more concrete example +----------------------- + + + bw1 contains html1 which includes css1, css2, img1, img2 + + bw2 contains html2 which includes css1, img1, img2 + + bw3 contains img1 + + Neither HTML nor CSS contents are shareable. + All shareable contents are requested from the high-level cache + once their type is known. + + Low-level cache contains source data for: + + 1 - html1 + 2 - html2 + 3 - css1 + 4 - css2 + 5 - img1 + 6 - img2 + + High-level cache contains: + + Content objects (ll-handle in parentheses): + + + c1 (1 - html1) + + c2 (2 - html2) + + c3 (3 - css1) + + c4 (4 - css2) + + c5 (5 - img1) + + c6 (6 - img2) + + c7 (3 - css1) + + Content handles (objects in parentheses): + + + h1 (c1, used by bw1) + + h2 (c3, used by h1) + + h3 (c4, used by h1) + + h4 (c2, used by bw2) + + h5 (c7, used by h4) + + h6 (c5, used by h1,h4,bw3) + + h7 (c6, used by h1,h4) + + If img1 was not of known type when requested: + + Content handles (objects in parentheses): + + + h1 (c1, used by bw1) + + h2 (c3, used by h1) + + h3 (c4, used by h1) + + h4 (c2, used by bw2) + + h5 (c7, used by h4) + + h6 (c5, used by h1) + + h7 (c6, used by h1,h4) + + h8 (c5, used by h4) + + h9 (c5, used by bw3) + +This achieves the desired effect that: + + + source data is shared between contents + + content objects are only created when absolutely necessary + + content usage/dependency is tracked and cycles avoided + + offline storage is possible + +Achieving this requires the use of indirection objects, but these are expected +to be small in comparison to the content objects / ll-cache objects that they +are indirecting. +

13 years, 8 months

1
0
0 / 0

r9560 MarkieB - /branches/MarkieB/gtkmain/Docs/BUILDING-AmigaCross

by netsurf＠semichrome.net

Author: MarkieB Date: Tue Sep 8 11:30:36 2009 New Revision: 9560 URL: http://source.netsurf-browser.org?rev=9560&view=rev Log: some advances in the Amiga cross-compile Modified: branches/MarkieB/gtkmain/Docs/BUILDING-AmigaCross Modified: branches/MarkieB/gtkmain/Docs/BUILDING-AmigaCross URL: http://source.netsurf-browser.org/branches/MarkieB/gtkmain/Docs/BUILDING-... ============================================================================== --- branches/MarkieB/gtkmain/Docs/BUILDING-AmigaCross (original) +++ branches/MarkieB/gtkmain/Docs/BUILDING-AmigaCross Tue Sep 8 11:30:36 2009 @@ -2,10 +2,45 @@ http://utilitybase.com/article/show/2007/06/23/231/Installing+an+AmigaOS+... -a more Mac-oriented article is at +a more Mac-oriented article [though of potentially general utility] is at http://utilitybase.com/article/show/2006/05/21/188/Building+Amiga+OS+4+GC... more background at http://cross.zerohero.se/os4.html +cross-compile additional libs/tools +SDK +http://www.hyperion-entertainment.biz/ + +newlib +http://sources.redhat.com/newlib/ + +clib2 +http://sourceforge.net/projects/clib2/ + +ixemul +http://strohmayer.org/sfs/ + +libnix +http://sourceforge.net/projects/libnix/ + +though newlib / clib2 are apparently already included in the ppc-amigaos-gcc tarball + then install linked libs in the correct place + +zlib +libxml +regex [unless in clib2] +libcurl +libiconv +openssl +libpng +libmng +liblcms +libjpeg + +libparserutils +libhubbub +libcss +libnsbmp +libnsgif

13 years, 8 months

1
0
0 / 0

r9559 MarkieB - /branches/MarkieB/windows/Docs/BUILDING-Windows

by netsurf＠semichrome.net

Author: MarkieB Date: Tue Sep 8 11:29:57 2009 New Revision: 9559 URL: http://source.netsurf-browser.org?rev=9559&view=rev Log: add libcss instructions Modified: branches/MarkieB/windows/Docs/BUILDING-Windows Modified: branches/MarkieB/windows/Docs/BUILDING-Windows URL: http://source.netsurf-browser.org/branches/MarkieB/windows/Docs/BUILDING-... ============================================================================== --- branches/MarkieB/windows/Docs/BUILDING-Windows (original) +++ branches/MarkieB/windows/Docs/BUILDING-Windows Tue Sep 8 11:29:57 2009 @@ -124,6 +124,12 @@ [export GCCSDK_INSTALL_ENV=/usr/local/mingw] should be unnecessary at this stage $ make TARGET=windows $ make install TARGET=windows + +libcss +svn://svn.netsurf-browser.org/trunk/libcss +[export GCCSDK_INSTALL_ENV=/usr/local/mingw] +make TARGET=windows +make install TARGET=windows openssl: may be problematic; retrieve the most recent version from the project homepage

13 years, 8 months

1
0
0 / 0

r9558 jmb - /trunk/libsvgtiny/include/svgtiny.h

by netsurf＠semichrome.net

Author: jmb Date: Tue Sep 8 05:06:33 2009 New Revision: 9558 URL: http://source.netsurf-browser.org?rev=9558&view=rev Log: Fix colour component macros on RISC OS. Modified: trunk/libsvgtiny/include/svgtiny.h Modified: trunk/libsvgtiny/include/svgtiny.h URL: http://source.netsurf-browser.org/trunk/libsvgtiny/include/svgtiny.h?rev=... ============================================================================== --- trunk/libsvgtiny/include/svgtiny.h (original) +++ trunk/libsvgtiny/include/svgtiny.h Tue Sep 8 05:06:33 2009 @@ -12,7 +12,7 @@ typedef int svgtiny_colour; #define svgtiny_TRANSPARENT 0x1000000 -#ifdef riscos +#ifdef __riscos__ #define svgtiny_RGB(r, g, b) ((b) << 16 | (g) << 8 | (r)) #define svgtiny_RED(c) ((c) & 0xff) #define svgtiny_GREEN(c) (((c) >> 8) & 0xff)

13 years, 8 months

1
0
0 / 0

r9557 MarkieB - /branches/MarkieB/gtkmain/gtk/gtk_window.c

by netsurf＠semichrome.net

Author: MarkieB Date: Tue Sep 8 03:19:08 2009 New Revision: 9557 URL: http://source.netsurf-browser.org?rev=9557&view=rev Log: tidy up some todo wrt mouse clicks Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c URL: http://source.netsurf-browser.org/branches/MarkieB/gtkmain/gtk/gtk_window... ============================================================================== --- branches/MarkieB/gtkmain/gtk/gtk_window.c (original) +++ branches/MarkieB/gtkmain/gtk/gtk_window.c Tue Sep 8 03:19:08 2009 @@ -456,25 +456,18 @@ g->mouse->pressed_x = event->x / g->bw->scale; g->mouse->pressed_y = event->y / g->bw->scale; - if (event->button == 3) { - /** \todo - * Firstly, MOUSE_PRESS_2 on GTK is a middle click, which doesn't - * appear correct to me. Secondly, right-clicks are not passed to - * browser_window_mouse_click() at all, which also seems incorrect - * since they should result in browser_window_remove_caret(). - * - * I would surmise we need a MOUSE_PRESS_3, unless right-clicking is - * supposed to be mapped to MOUSE_PRESS_2, but that doesn't appear - * correct either. - */ + switch (event->button) { + case 1: + g->mouse->state = BROWSER_MOUSE_PRESS_1; + break; + case 3: browser_window_remove_caret(g->bw); - nsgtk_scaffolding_popup_menu(g->scaffold, g->mouse->pressed_x, g->mouse->pressed_y); - return TRUE; - } - - switch (event->button) { - case 1: g->mouse->state = BROWSER_MOUSE_PRESS_1; break; - case 2: g->mouse->state = BROWSER_MOUSE_PRESS_2; break; + nsgtk_scaffolding_popup_menu(g->scaffold, g->mouse->pressed_x, + g->mouse->pressed_y); + g->mouse->state = BROWSER_MOUSE_PRESS_2; + break; + default: + return FALSE; } /* Handle the modifiers too */ if (event->state & GDK_SHIFT_MASK)

13 years, 8 months

1
0
0 / 0

r9556 MarkieB - /branches/MarkieB/gtkmain/gtk/gtk_window.c

by netsurf＠semichrome.net

Author: MarkieB Date: Tue Sep 8 02:46:39 2009 New Revision: 9556 URL: http://source.netsurf-browser.org?rev=9556&view=rev Log: allow straight pointer moves Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c URL: http://source.netsurf-browser.org/branches/MarkieB/gtkmain/gtk/gtk_window... ============================================================================== --- branches/MarkieB/gtkmain/gtk/gtk_window.c (original) +++ branches/MarkieB/gtkmain/gtk/gtk_window.c Tue Sep 8 02:46:39 2009 @@ -408,7 +408,7 @@ struct gui_window *g = data; bool shift = event->state & GDK_SHIFT_MASK; bool ctrl = event->state & GDK_CONTROL_MASK; - if ((abs(event->x - g->last_x) < 5) || (abs(event->y - g->last_y) < 5)) + if ((abs(event->x - g->last_x) < 5) && (abs(event->y - g->last_y) < 5)) /* necessary for touch screens */ return FALSE; if (g->mouse->state & BROWSER_MOUSE_PRESS_1){

13 years, 8 months

1
0
0 / 0

r9555 MarkieB - /branches/MarkieB/gtkmain/gtk/gtk_window.c

by netsurf＠semichrome.net

Author: MarkieB Date: Tue Sep 8 02:28:56 2009 New Revision: 9555 URL: http://source.netsurf-browser.org?rev=9555&view=rev Log: make gtk touchscreen sensitivity acceptable - credit Graham Gower Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c Modified: branches/MarkieB/gtkmain/gtk/gtk_window.c URL: http://source.netsurf-browser.org/branches/MarkieB/gtkmain/gtk/gtk_window... ============================================================================== --- branches/MarkieB/gtkmain/gtk/gtk_window.c (original) +++ branches/MarkieB/gtkmain/gtk/gtk_window.c Tue Sep 8 02:28:56 2009 @@ -279,6 +279,7 @@ GDK_BUTTON_PRESS_MASK | GDK_BUTTON_RELEASE_MASK | GDK_POINTER_MOTION_MASK | + GDK_POINTER_MOTION_HINT_MASK | GDK_KEY_PRESS_MASK | GDK_KEY_RELEASE_MASK); GTK_WIDGET_SET_FLAGS(GTK_WIDGET(g->drawing_area), GTK_CAN_FOCUS); @@ -407,7 +408,9 @@ struct gui_window *g = data; bool shift = event->state & GDK_SHIFT_MASK; bool ctrl = event->state & GDK_CONTROL_MASK; - + if ((abs(event->x - g->last_x) < 5) || (abs(event->y - g->last_y) < 5)) + /* necessary for touch screens */ + return FALSE; if (g->mouse->state & BROWSER_MOUSE_PRESS_1){ /* Start button 1 drag */ browser_window_mouse_click(g->bw, BROWSER_MOUSE_DRAG_1,

13 years, 8 months

1
0
0 / 0

r9554 chris_y - /trunk/netsurf/amiga/thumbnail.c

by netsurf＠semichrome.net

Author: chris_y Date: Mon Sep 7 17:55:31 2009 New Revision: 9554 URL: http://source.netsurf-browser.org?rev=9554&view=rev Log: ensure current_redraw_browser is set Modified: trunk/netsurf/amiga/thumbnail.c Modified: trunk/netsurf/amiga/thumbnail.c URL: http://source.netsurf-browser.org/trunk/netsurf/amiga/thumbnail.c?rev=955... ============================================================================== --- trunk/netsurf/amiga/thumbnail.c (original) +++ trunk/netsurf/amiga/thumbnail.c Mon Sep 7 17:55:31 2009 @@ -40,9 +40,11 @@ bitmap->nativebmwidth = bitmap->width; bitmap->nativebmheight = bitmap->height; ami_clearclipreg(&browserglob); + current_redraw_browser = curbw; plot = amiplot; content_redraw(content, 0, 0, content->width, content->width, 0, 0, content->width, content->width, 1.0, 0xFFFFFF); + current_redraw_browser = NULL; if(GfxBase->lib_Version >= 53) // AutoDoc says v52, but this function isn't in OS4.0, so checking for v53 (OS4.1) {

13 years, 8 months

1
0
0 / 0

r9553 MarkieB - /branches/MarkieB/gtkmain/gtk/res/

by netsurf＠semichrome.net

Author: MarkieB Date: Mon Sep 7 12:31:11 2009 New Revision: 9553 URL: http://source.netsurf-browser.org?rev=9553&view=rev Log: avoid committing toolbar customization Modified: branches/MarkieB/gtkmain/gtk/res/ (props changed) Propchange: branches/MarkieB/gtkmain/gtk/res/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Sep 7 12:31:11 2009 @@ -1,0 +1,1 @@ +toolbarIndices

13 years, 8 months

1
0
0 / 0

r9552 MarkieB - /branches/MarkieB/windows/windows/res/

by netsurf＠semichrome.net

Author: MarkieB Date: Mon Sep 7 12:30:06 2009 New Revision: 9552 URL: http://source.netsurf-browser.org?rev=9552&view=rev Log: avoid committing changes to prefs file Modified: branches/MarkieB/windows/windows/res/ (props changed) Propchange: branches/MarkieB/windows/windows/res/ ------------------------------------------------------------------------------ --- svn:ignore (added) +++ svn:ignore Mon Sep 7 12:30:06 2009 @@ -1,0 +1,1 @@ +preferences

13 years, 8 months

1
0
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

netsurf-commits September 2009