On Mon, 23 Jun 2014 17:58:52 +0100, Peter Young wrote: I'm sure Vince will correct me here, but I believe NetSurf saves the cache files to disk when they are downloaded (as opposed to when they are evicted from the memory cache), so you will get a delay as NetSurf gets busy saving the files. If you've gone back to the same site though most of the files should be loading from disk and I wouldn't expect any additional delay. You will need to post a log file so we can get a handle on exactly what is causing the pauses. Yes. That's only 22MB. The default limit is 1GB. There's no point in backing up these files. Chris

On Mon, 23 Jun 2014 21:13:17 +0100, Peter Young wrote: The best thing to do is raise a bug report and attach it to that. Chris

On Mon, 23 Jun 2014 22:43:52 +0100, Peter Young wrote: If it's not working as it should, it's a bug. :) There may be something that can be done to improve matters, such as delaying disk cache writes until the browser isn't busy, or only writing out when the item is removed from the memory cache. It may be that it is a true bug and is writing when it shouldn't be, or there is some performance problem which is nothing to do with disk I/O. Chris

Peter Young, on 23 Jun, wrote: I have found much the same, a really good example of this is the Daily Mail's heavy weight site. http://www.dailymail.co.uk/home/index.html Writes to the Raspberry Pi's SD Card are so slow that !Cache is not going to be good news on it. It is better with !Cache on a Fat32 harddisc connected to the Pi and on the Iyonix but is still an issue. Overall I was not persuaded that the cache results is any meaningful speed up and could even slow things up, not just on the Raspberry Pi but also on the Iyonix and VRPC on a Windows 7 laptop with an SSD. A lot of stuff is cached and the default maximum cache size is 1GB. It's not worth backing up, it's transient data that expires in a default of 28 days. I have uninstalled !Cache. -- David Pitt

On Mon, Jun 23, 2014 at 07:04:53PM +0100, David Pitt wrote: Ultimately, my advice is to not visit this service. This stands regardless of any cache issues that may exist :) Indeed, SD has poor write performance almost anywhere, like most flash-based devices. Certainly on UNIX and BeOS, it seems to provide a significant performance boost, but this is probably because of their far superior IO layers. On RISC OS, the disc cache *may* only be a win for people on slow connections. B.

Malcolm Hussain-Gambles, on 23 Jun, wrote: The Panda is going to be faster than a Raspberry Pi. As I understand it the small file write speed even on supposedly fast SD cards can be very slow. This certainly seems to be the case on the Pi. It first became really apparent after a particularly long winded !PackMan installation. A large number of small files is a problem. External drives are better but are only USB1. !Cache on a RamDisc looks much more promising. This is much better, the improvement is clear, and the machine is not taken over as the cache is written. -- David Pitt

Daniel Silverstone, on 24 Jun, wrote: I had not got that far. The notion was just to highlight where the blockage is. In practice the two options perform similarly, as very briefly tested. -- David Pitt

Tony Moore, on 24 Jun, wrote: Indeed. I do agree, I was just testing something. -- David Pitt

Rob Kendrick, on 23 Jun, wrote: [snip] !Cache on a RamDisc confirms that. -- David Pitt

Although this reply is to Peter it applies to all the subsequent discussion. Firstly, as I did memntion in my original mail this feature is new and not tuned yet so may have adverse behaviour on some systems. Please do not draw any conclusions about the usefulness or otherwise of this feature from *development* snapshots. If you want stable behaviour the 3.1 release should be used. It may well turn out that this feature is simply unsuitable for RISC OS but we are at the beginning of a long road. As we have seen with the issues around !Cache and now with general perfomance, the challenges of building a cache suitable for use across many systems are not inconsiderable On Mon, Jun 23, 2014 at 05:58:52PM +0100, Peter Young wrote: Subsequent mails indicate you were using this correctly, as to the "worth it" that is what we need to evaluate. I prevoiusly mentioned it, but I shall re-iterate: The persistant cache is purely a trade, in this case it is trading disc resource for network resources. I use the term disc resource carefully, it includes both storage space *and* the time to store and retrieve the files. Similarly network resource is the data downloaded *and* the latency getting that data. What is happening here is that when your browser has gone idle after retrieving the website you have numerous objects (images html files etc.) which are eligible to be written out to the persistent cache. Given the front page of the BBC news site is (right now) 1.1 Megabytes in 187 files which can rise to in the region of 2 Megabytes or more if there are a image heavy news stories, that represents a great deal of data to be written out. The write out task then starts writing these to disc limited by a bandwidth cap. In the current implementation this is hard wired to a maximum write bandwidth of 512K/second. It had not occurred to me that such a rate would not be achievable. Because the code does not moderate its write rate you have to watch the hourglass as it saves those files to disc. This is made much worse, as as I understand it from knowlageable RISC OS people, because disc writes are not performed in the background. In your example if we assume you managed to get four megabytes of cached data to be written and it took 30 seconds to achive that, we get a write rate of around 130K/second or roughly a Megabit/second. Your network connection does not have to be very good at all to outrun the disc and hence the disc cache is making a horrible trade in your case. The web is huge these days, 22 Megabytes is literally only 20 pages on most sites, even if the site shares those objects between multiple pages, just visiting all the heading sections of BBC news requires 10Megabytes of persistant storage and over 1000 files. For example google chrome by default uses 16Megabyte blocks to store sub 16k sized objects and creates four of those to start with It is not uncommon for a well used browser to have several gigabytes of disc used for web caches. In most current systems using a gigabyte of RAM let alone Disc is not uncommon. In summary, the cache: - Is still in development. - Is not a panacea and will not benefit everyone. - Is a compromise trade, and it seems for some systems with slow disc it is literally faster to retrieve from network than from local storage. - Is likely to be very large to be effective as the source web pages are large. - Can be disabled by setting its size to zero. I will look into adding a heuristic to disable or at least tune the cache writeout if it detects it is exceeding the available disc bandwidth. -- Regards Vincent http://www.kyllikki.org/

On Tue, Jun 24, 2014 at 06:36:18PM +0100, Chris Young wrote: Assuming you were running it on one of the flavours of UNIX available for it (Linux, NetBSD), then no. These operating systems receive the write requests from applications and queue them for writing to underlying block devices in the background, while other apps sit there waiting for input or idling. Under RISC OS, file system writes stop /everthing/ until they complete. (Under UNIX, applications should call the fdatasync() function when the data stored to disc is in a consistent/useful state, which guarentees the data has reached backing store, but then they also have pre-emptive multitasking so you can still use your computer when this happens.) B.

On Wed, Jun 25, 2014 at 12:14:16PM +0100, george greenfield wrote: It means that it writes smaller chunks and calls Wimp_Poll in between them to give other applications a chance, at the expense of expediency. The whole system stops while the individual chunks are written, being accepted by FileSwitch, which then in turn passes them to the handling file system (most likely FileCore), which then hands them to the block device driver (ADFS, IDEFS, SDFS, etc), percolating the success/failure result back up the stack to the application. On other operating systems (UNIX, Windows, BeOS, AmigaOS, OS X), the equivalent of FileSwitch passes the request onto the file system which then writes the data through a buffering system and thus can return almost straight away, while a background process in the kernel then flushes the buffer to the block device while the system is not busy, or a timeout expires. During this process, reads are satisfied through the buffer because they've not yet reached the block device. There is some functionality in Vince's disc cache work to allow for something similar to the "file operations multitask" option, and may need tuning. As he said, this is an experimental feature and the feedback that has been happening does have some value. B.

On Wed, Jun 25, 2014 at 01:32:58PM +0100, Steve (plusnet) wrote: But even with background transfers, the Wimp will typically not continue, no? My understanding is that OS_File SWIs that write data only return when the data is written, and thus the application cannot call Wimp_Poll to allow multitasking to continue - and even if you're using UnixLib threads this doesn't help you because threading does not continue over such SWIs. (Getting into the technical distinctions between WIMP and non-WIMP tasks was something I was trying to avoid getting into on the user's mailing list :) B.

On Wed, Jun 25, 2014 at 01:44:01PM +0100, Steve (plusnet) wrote: Of course not :) But from the user's point of view, everything does stop :) B.

On 25 Jun at 13:32, "Steve (plusnet)" <steve(a)revi11.plus.com&gt; wrote: <snip of technical discussion> I have intermittently subscribed to a bounty or two. I wonder if the ROOL systems could improve the volume of subscriptions received: 1. By issuing a public report occasionally (yearly? And to csaa?) on what has been achieved and what could do with more funding. 2. When a Bounty has been fulfilled, write to the subscribers to tell them of this good news and invite them to subscribe to other bounties instead, even including with this a list of priorities for RISC OS. 3. Consider changing some aspects of ROOL to a charity, certainly including the Bounty aspect. This would enable the trustees to reclaim income tax paid by the bounty donors. Further the donors who pay higher rate tax could deduct their subscription from the higher rate income, thereby reducing their higher rate tax; if they felt so minded they could then increase their donation by the amount of this higher rate tax relief. That said, how much do you think is needed in the relevant bounty (which?) to encourage a developer to start work? -- Tim Powys-Lybbe tim(a)powys.org for a miscellany of bygones: http://powys.org/

On 25 Jun at 15:22, "Steve (plusnet)" <steve(a)revi11.plus.com&gt; wrote: Can't stand forums. Too much hassle to find the site and the discussion. Mailing lists and usenet are so much easier: the discussion just appears on MessengerPro. I take it that you are not interested in following up your comments on bounties, to which I was merely replying? -- Tim Powys-Lybbe tim(a)powys.org for a miscellany of bygones: http://powys.org/

On 24 Jun 2014 Vincent Sanders <vince(a)netsurf-browser.org&gt; wrote: [snip very interesting bits, but a lot of which goes right over my head!] Many thanks for this, and I did realise that it's still rather a test feature and a compromise, but I just wondered whether I was doing things right, and I'm glad I am. I'm also glad the my logfile was of interest. I'll stick with it (like many people of my age, I find I get more patient as I get older), and will await developments with interest. Best wishes, Peter. -- Peter Young (zfc W) and family Prestbury, Cheltenham, Glos. GL52, England http://pnyoung.orpheusweb.co.uk pnyoung(a)ormail.co.uk