1:49 p.m.
We've been having some discussion on IRC over the current mimetype detection...
There are several issues with the current call:
- it assumes a local file, so implementations try to read xattrs,
- it returns "text/html" on failure instead of NULL, which have some side
effects.
In several places I would need some similar calls but for urls or data buffers but I
can't use the existing call correctly.
For example, with gopher some item types don't have a specific mime type so it must be
probed either by extension or by sniffing the incoming data. But using fetch_filetype()
which returns text/html by default makes it try to display binary files...
BeOS had a mime sniffer for 15 years, and other "sane" OS have one too.
XDG seems to have something specified as well:
http://www.freedesktop.org/wiki/Specifications/shared-mime-info-spec
There seems to be an attempt at normalizing some mime sniffing as part of HTML5 also:
http://code.google.com/p/mimesniff/
http://tools.ietf.org/html/draft-abarth-mime-sniff-03
What I'd propose is to have something like:
[const?] char *fetch_mime_localfile(const char *path);
[const?] char *fetch_mime_by_ext(const char *filename);
[const?] char *fetch_mime_by_data(const void *data, size_t size);
The last 2, the new ones, are easily implemented in BeOS for ex. with the
BMimeType:GuessMimeType() variants:
http://cvincent.pagesperso-orange.fr/bebook/Release Notes/StorageKit.html#GuessMimeType()
François.
10:03 p.m.
On Sat, 14 May 2011 20:49:10 +0200, François Revol wrote:
[const?] char *fetch_mime_by_ext(const char *filename);
Personally I hate detecting filetypes by filename/extension - it's
prone to errors/hacking. Look at all the problems on Windows,
renaming a file makes it think the filetype has changed, you can trick
things like Outlook so easily just by renaming files.
I also realise that sometimes the filename is the only way to
determine a filetype. I think an "integrated" approach would be
better, one function which passes the filename and the data. I see no
reason why the data (or at least the first few bytes) couldn't be
available before you need to work out what type it is.
Chris
7:48 a.m.
Le 30 mai 2011 à 05:03, Chris Young a écrit :
On Sat, 14 May 2011 20:49:10 +0200, François Revol wrote:
> [const?] char *fetch_mime_by_ext(const char *filename);
Personally I hate detecting filetypes by filename/extension - it's
prone to errors/hacking. Look at all the problems on Windows,
renaming a file makes it think the filetype has changed, you can trick
things like Outlook so easily just by renaming files.
+1, that's why we use MIME types on files in xattrs and have a MIME sniffer in Haiku,
just like BeOS did for 15 years. :p
I also realise that sometimes the filename is the only way to
determine a filetype. I think an "integrated" approach would be
better, one function which passes the filename and the data. I see no
reason why the data (or at least the first few bytes) couldn't be
available before you need to work out what type it is.
Sometimes the file is empty :p
Well, there are several cases:
- existing files (file:) that can have a mime xattr, the BeOS port already checks them for
fetch_filetype,
- virtual files (not yet downloaded urls or in-memory cached data not downloaded) where
the OS didn't yet sniff the mime, for which we have to sniff ourselves.
Ideally we would:
- check if the file is real, and it has a MIME info,
- try to sniff it,
- fallback to extension matching.
The API doesn't have to provide separate functions indeed, it can be a global one
which can have the data buffer be optional, if NULL then we just don't sniff.
Of course there are some corner cases like directories (BeOS also has a mime type for
those but different than NS'), and usual types like text/html which the OS might
differentiate a bit too much sometimes or identify differently (text/xml+html for
xhtml?).
For ex, Haiku uses text/x-source-code and usually doesn't differentiate between
python, perl, bash, C++ or whatever source language.
François.
11:17 a.m.
On Mon, 30 May 2011 14:48:20 +0200, François Revol wrote:
+1, that's why we use MIME types on files in xattrs and have a
MIME
sniffer in Haiku, just like BeOS did for 15 years. :p
Things are a little more difficult here as there is no OS-wide MIME
support, so I have to detect the file type and then convert it myself
to the equivalent MIME type. I've mostly rewritten it to clean it up,
just need to migrate the old functions/code across to the new scheme.
> I see no
> reason why the data (or at least the first few bytes) couldn't be
> available before you need to work out what type it is.
Sometimes the file is empty :p
In that case, the MIME type is application/unknown or something like
that :)
Of course there are some corner cases like directories (BeOS also
has a mime type for those but different than NS')
I don't think NS uses that any more.
, and usual types
like text/html which the OS might differentiate a bit too much
sometimes or identify differently (text/xml+html for xhtml?).
One of the documents you linked to provides all the test cases for
different (usual) filetypes and suggested not to do any other tests
for these types. Perhaps files should be sniffed by the core using
this citeria and only passed to the platform code if nothing matches?
Chris
4317
days inactive
4333
days old
netsurf-dev@netsurf-browser.org
3 comments
2 participants
participants (2)
-
Chris Young -
François Revol