PassMark Logo
Home » Forum

cyrillic, windows -1251 and UTF-8

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • Gru
    Junior Member
    • Nov 2005
    • 2
    #1

    cyrillic, windows -1251 and UTF-8

    Hello everybody,


    I want to index webpage’s that includes cyrllic (<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="ru">) as UTF-8? Is it possible somehow? I tried to index it as UTF-8 directly but it didn’t work of course.

    The problem is that Im going to put some of the result in a mysql-db and it would be a lot easier if I get the result in UTF-8. Especially since Im indexing several webpage’s including different languages.

    I do not have access to all the webpage’s that I’m trying to index (can’t edit them)...

    Does someone have suggestions how to solve this problem or to proceed?

    Regards
    J Gru
    Tags: None
    • David
      Administrator
      • Dec 2004
      • 4714
      #2
      If everything is in UTF-8 it should work. (Zoom settings, the pages being indexed and the actual search page)

      This FAQ page contains information for using Zoom with non-english languages. It includes a section on Russian cyrllic,
      http://www.wrensoft.com/zoom/support/languages.html

      -----
      David

        Comment

        • Gru
          Junior Member
          • Nov 2005
          • 2
          #3
          Most of the pages beeing indexed are not UTF-8, they are windows-1251 unfortunately...


          But tnx for such a fast reply.

          /Gru

            Comment

            • Ray
              Administrator
              • Dec 2004
              • 4357
              #4
              The current version of Zoom requires that the input charset/encoding of the pages being indexed must be the same as the encoding for the resulting search page and index files.

              However we have plans to change this in the next major release (V5.0) by allowing the user to specify a different encoding for the pages being scanned in, and the output index files. It will also be able to automatically determine the charset of a web page (when specified) so you will be able to index multiple websites - with different encodings, and have a single UTF-8 search page for them. V5.0 will be released some time next year.
              --Ray
              Wrensoft Web Software
              Sydney, Australia
              Zoom Search Engine

                Comment

                Previous template Next
                Working...