SPHINX SEARCH ENGINE CLIENT API USAGE

          General approach for querying Sphinx is similar in all of them.

                           (Java, PHP, C ,Perl, python )

1. create a sphinx client object.
2. set query options.
3. set query.
4. connect to sphinx server (if not connected) .
5. send query.
6. receive results.
7. close connection
   

PHP Tutorials:

http://www.php.net/manual/en/sphinxclient.query.php.

Java Tutorials:

http://cmusphinx.sourceforge.net/sphinx4/.

Perl Tutorials:

http://cpansearch.perl.org/src/JJSCHUTZ/Sphinx-Search-0.27.2/README.

C Tutorials:

https://github.com/johnl/deb-sphinx-search/tree/9e16b906e8dae79d131950c5d674f40d352ed24e/api/libsphinxclient.

Python Tutorials:

http://nesv.blogspot.in/2011/06/sphinx-generating-api-documentation-for.html#!/2011/06/sphinx-generating-api-documentation-for.html.


                                                                                                                        -PAVANKUMAR JOSHI

SPHINX SEARCH ENGINE CLIENT API'S

For PHP:

Download sphinxapi.php library from

http://code.google.com/p/sphinxsearch/source/browse/trunk/api/sphinxapi.php?r=2014


For Java:
Download Sphinx-4 library and more information (Usage with example) from
http://cmusphinx.sourceforge.net/wiki/sphinx4:howtobuildand_run_sphinx4#downloading_sphinx-4 .

For Perl:
Download sphinx_api_0.9.7-rc1.tar.gz from http://www.ilance.nl/sphinx_perl_api.

For C:
Download libsphinxclient library from
http://rpm.pbone.net/index.php3?stat=3&search=libsphinxclient&srodzaj=3.


For Ruby :
Download sphinxclientapi library from
http://kpumuk.info/projects/ror-plugins/sphinx/#download.

For Python:

Download sphinxapi.py library from
http://code.google.com/p/sphinxsearch/source/browse/trunk/api/sphinxapi.py?r=2011.

                                                                                         -PAVANKUMAR JOSHI

SPHINX SEARCH ENGINE USAGE

On Linux: (On command Line)

1. Create a configuration file.

Default configuration file name is sphinx.conf.

All Sphinx programs look for this file in current working directory by default.

Sample configuration file, sphinx.conf.dist, which has all the options

documented, is created by configure. Copy and edit that sample file to

make your own configuration:

(assuming Sphinx is installed into /usr/local/sphinx/).

$ cd  /usr/local/sphinx/etc

$ cp sphinx.conf.dist sphinx.conf

$ vi  sphinx.conf

 

Sample configuration file is setup to index documents table from MySQL

database test; so there's example.sql sample data file to populate that table with a

few documents for testing purposes:

$ mysql -u test < /usr/local/sphinx/etc/example.sql.

2. Run the indexer to create full-text index from your data:

$ cd /usr/local/sphinx/etc

$ /usr/local/sphinx/bin/indexer –all

 

3. Query your newly created index!

To query the index from command line, use search utility:

$ cd /usr/local/sphinx/etc

$ /usr/local/sphinx/bin/search test

On Windows: (On command Line)

Step 1. Configure Sphinx:

1. Open the extension archive (e.g. SphinxSearch-trunk-r97051.tar.gz) and
extract sphinx.conf to the Sphinx folder (C:\Sphinx). 


2. Edit the contents of C:\Sphinx\sphinx.conf. Look for sql_db, sql_user,
sql_pass, path, log, query_log and pid_file variables.
Create the paths as necessary. Make sure you use backslashes (\) instead of
forward slashes (/). 


3. Don't forget to add a binlog_path = line (that points to a real folder) in the
searchd to make sure that the service starts successfully in windows



 

Step 2: Run Sphinx Indexer :


Running an incremental indexing process in command shell mode
Run the sphinx indexer to prepare for searching:


        C:\Sphinx\bin\indexer --config C:\Sphinx\sphinx.conf --all

Once again, make sure to replace the paths to match your installation. This
process is actually pretty fast, but clearly depends on how large your wiki is.
Just be patient and watch the screen for updates.




Step 3: Search:


Actual search process from Mediawiki displayed in command shell mode
When the indexer is finished, test that sphinx searching is actually working:


        C:\Sphinx\bin\search --config C:\Sphinx\sphinx.conf "search string"

                                                                                    -PAVANKUMAR JOSHI

Time Management Tips

Here are 15 practical time management tips to help you get started…

1. Write things down
A common time management mistake is to try to use your memory to keep track of
too many details leading to information overload. Using a to-do list to write
things down is a great way to take control of your projects and tasks and keep
you organized.

2. Prioritize your list
Prioritizing your to-do list helps you focus and spend more of your time on the
things that really matter to you. Rate your tasks into categories using the ABCD
prioritization system described in the time management course.

3. Plan your week
Spend some time at the beginning of each week to plan your schedule. Taking the
extra time to do this will help increase your productivity and balance your
important long-term projects with your more urgent tasks. All you need is
fifteen to thirty minutes each week for your planning session.

4. Carry a notebook
You never know when you are going to have a great idea or brilliant insight.
Carry a small notebook with you wherever you go so you can capture your
thoughts. If you wait too long to write them down you could forget. Another
option is to use a digital recorder.

5. Learn to say no
Many people become overloaded with too much work because they overcommitted;
they say yes when they really should be saying no. Learn to say no to low
priority requests and you will free up time to spend on things that are more
important.

6. Think before acting
How many times have you said yes to something you later regretted? Before
committing to a new task, stop to think about it before you give your answer.
This will prevent you from taking on too much work.

7. Continuously improve yourself
Make time in your schedule to learn new things and develop your natural talents
and abilities. For example, you could take a class, attend a training program,
or read a book. Continuously improving your knowledge and skills increases your
marketability, can help boost your career, and is the most reliable path to
financial independence.

8. Think about what you are giving up to do your regular activities
It is a good idea to evaluate regularly how you are spending your time. In some
cases, the best thing you can do is to stop doing an activity that is no longer
serving you so you can spend the time doing something more valuable. Consider
what you are giving up in order to maintain your current activities.

9. Use a time management system
Using a time management system can help you keep track of everything that you
need to do, organize and prioritize your work, and develop sound plans to
complete it.

10. Identify bad habits
Make a list of bad habits that are stealing your time, sabotaging your goals,
and blocking your success. After you do, work on them one at a time and
systematically eliminate them from your life. Remember that the easiest way to
eliminate a bad habit, it to replace it with a better habit. In these days it’s
a social networking addiction.

11. Don’t do other people’s work
Are you in the habit of doing other people’s work because of a ‘hero’
mentality? Doing this takes up time that you may not have. Instead, focus on
your own projects and goals, learn to delegate effectively, and teach others how
to do their own work.

12. Keep a goal journal
Schedule time to set and evaluate your goals. Start a journal and write down
your progress for each goal. Go through your goal journal each week to make sure
you are on the right track. Keeping a journal on your computer has never been
easier!

13. Don’t be a perfectionist
Some tasks don’t require your best effort. Sending a short email to a
colleague, for example, shouldn’t take any more than a few minutes. Learn to
distinguish between tasks that deserve to be done excellently and tasks that
just need to be done.

14. Beware of “filler” tasks
When you have a to-do list filled with important tasks, be careful not to get
distracted by “filler” tasks. Things such as organizing your bookcase or filing
papers can wait until you tackle the items that have the highest priority.

15. Avoid “efficiency traps”
Being efficient doesn’t necessarily mean that you are being productive. Avoid
taking on tasks that you can do with efficiency that don’t need to be done at
all. Just because you are busy and getting things done doesn’t mean you are
actually accomplishing anything significant.



                                                                                    -PAVANKUMAR JOSHI

BLACKBERRY CODE SIGNING STEPS

Why need code signing?

if use any of the controlled API's: Runtime API, Blackberry Apps API and Crypto API. These APIs need signing to be able to execute on a real BlackBerry device. Without signing can use these APIs only on a simulator.

Code signing is not required when applications are loaded and tested on a BlackBerry Smart phone Simulator.

Developers order code signing keys from RIM, and receive a set of files that are then installed on their development machine. The BlackBerry development tools are then able to provide these key files to the BlackBerry Signature Tool when the developer requests that their application be signed. The BlackBerry Signature applies a unique signature, representing the identity of the developer, to all *.cod files that compose a BlackBerry application. Developers should not share their code signing keys with others.

Signing Your Application Using the BlackBerry JDE Plug-In for Eclipse

As explained below, the complete steps for setting up and using BlackBerry code signing process are:

1. Order Code Signing Keys from RIM
2. Install Code Signing Keys
3. Sign Application using the BlackBerry development tools
   Order Code Signing Keys from RIM
   

Developers can https://www.blackberry.com/SignedKeys/nfc-form.html. As part of the registration process, developers must choose and provide a PIN that they will use while installing their code signing keys. BlackBerry Code Signing keys are free of charge.

Once registered, a set of 3 keys and installation instructions will each be delivered in a separate email. Code signing keys do not have any form of expiry, and can be used indefinitely to sign many BlackBerry applications.

Typically, registration for code signing keys is processed within 2 hours. If you haven’t received your keys within 2 hours of submitting the registration form, please check your spam filters, the keys will be addressed from websigner@ws-smtp.rim.net. If you are unable to locate them contact BlackBerry support at CodeSigningOrders@BlackBerry.com.

 

                                 Install Code Signing Keys

An email will be sent to the address used during the registration process.

This email will include attached code signing key files, and instructions for how these

files should be installed.

All code signing keys received need to be installed on the same PC. The same
password must be specified for all keys on the same PC. You should receive each
key/.csi file in a separate email message.
To register the attachment, please follow the instructions below:

Step 1: Save all 3 .csi files, sent via email, into the same directory.

(RBB.csi, RCR.csi & RRT.csi).

Step 2 : Start Eclipse.


Step3 : Click on the Window menu item -> Preferences.
Expand the BlackBerry Java Plug-in and select "Signiture Tool...".
And Click “Install New”

 



Step 4: Select one of the RBB .csi files saved in step 1 and click Open.

Step 5: Click "Yes" to create a new key pair file.



Step 6: Type a password for your private key of at least 8 characters, and type it again
to confirm. This is your private key password, which protects your private key.
Please remember this password as you will be prompted for it each time signing
is attempted or a signature key is installed.



Step 7: Move your mouse to generate date for a new private key.



Step 8: In the "Registration PIN" field, type the PIN number that you supplied when
purchasing signature keys.



Step 9: In the Private Key password field, type the password created in step 6.



Step 10 : Click "Register".



Step 11:Click "Exit".



Step 12: Repeat this process for the other csi files(RCR.csi & RRT.csi).

Sign Application using the BlackBerry Development Tools

Step 1: Right click on your project in the Package Explorer window

Step 2: Expand the "BlackBerry" menu item

Step 3: Select "Sign with Signature Tool"

Step 4: When prompted, enter the signing password you defined when installing your signature keys.

Once the code signing password has been provided, the development tool will launch the BlackBerry Signature Tool which will open in a new window.  You will see any necessary code signing keys applied to all COD files created as part of your application.

                                                                                   -PAVANKUMAR JOSHI

Blackberry Development Setup on Windows Machine

Blackberry JDE Installation

Required tools:

1. JDK 1.6+ : from http_://java.sun.com/javase/downloads/index.jsp.
   (Non-beta version).

32-bit or 64-bit depends on system requirements.

2. Blackberry JDE : from http://us.blackberry.com/developers/javaappdev/javadevenv.jsp.

(Latest non-beta version).

32-bit or 64-bit depends on system requirements.

Simulator comes on JDE Package itself. If get any issues on design and GUI issues. Try to

use different latest simulator.

3. Blackberry Simulator (Optional): From

http://us.blackberry.com/developers/resources/simulators.jsp

Installation steps:

1. Install JDK first.
2. After installing the JDK click on
   i. Control Panel -> System .
   ii. Click on Advanced tab and then click on “Environment variables”.
   iii. A dialog will popup.
   iv. Select the PATH variable and click Edit. Assuming installed JDK in the default path, add
   the line below in the text box that says “Variable value” :
   ;c:\program files\java\C:\Program Files\Java\jdk1.6.0_10\bin

If installed the JDK somewhere else or installed a different version, then change the path
accordingly. Make sure there is a semi-colon (;)at the beginning of the path you insert.

3. Install Blackberry JDE (double click on the installation file).

It will find the default JDK’s path from the environment variable “PATH”.

4. Click Start -> Research in motion -> JDE to run it.

                                          Known Issues:

When installing the blackberry JDE on 64-bit Windows, expect to run into several problems.  There are two big problems that I encountered when trying to get setup.

Problem 1 – Cannot find javaw
After running through the installation and trying to start the JDE, I received an error message saying ‘cannot find javaw’. That was odd to me because I was sure I had JDK installed. The problem was that the Blackberry JDE is not compatible with the 64-bit version of JDK, which is what I had. However, after downloading the 32-bit version of JDK, I found a second problem.

Problem 2 – Cannot find RIMIDEWin32Util.dll
Now I was confused. I was certain I had everything installed properly, but I was still receiving errors. A few forum posts suggested setting path and javahome to their full locations. Open up the following file:
C:\Program Files (x86)\Research In Motion\BlackBerry JDE 6.0.0\bin\ide.bat
Once opened we are going to add 2 lines to the beginning of the file. Change the file from this:
start javaw -Xmx1024M -DRIM_RUNTIME=..\lib -DRIM_BIN=..\bin -jar IDE.jar
to this:
SET PATH=C:\Program Files (x86)\Java\jdk1.6.0_22\bin;%PATH%
SET JAVAHOME=C:\Program Files (x86)\Java\jdk1.6.0_22
start javaw -Xmx1024M -DRIM_RUNTIME=..\lib -DRIM_BIN=..\bin -jar IDE.jar
Save, close and try to start the JDE. This time you should be error free.

Setting Up A Development Environment: Eclipse With Plug-in For BlackBerry

Method 1: Complete package (Eclipse + Blackberry)

I found the easiest way to quickly get set-up is to download the all-in-one ‘BlackBerry Java Plug-in’ (currently at version 1.5.0) (basically Eclipse IDE with a plug-in) from  .http://swdownloads.blackberry.com/Downloads/contactFormPreload.do?code=DC727151E5D55DDE1E950767CF861CA5&dl=E9E5F98CE5169068BE7B19B91F66BF08&check1=A

Windows Only: download the 32-bit version even if you have 64-bit Windows

Method 2: from Eclipse Update site

Required Tools:

1. Eclipse from http_://www.eclipse.org/downloads/ . - Preferred Helios or Classic(Latest version).

Installation(Eclipse Configuration) Steps:

1. Open Eclipse
2. Go Help -> Install New Softwares
3. Add Blackberry URL on work with field
   BlackBerry Update Site - http://www.blackberry.com/go/eclipseUpdate/3.6/java

   4. Add

  5. create New Repository

  6. Select Blackberry Java Plug-in(core) & any one Blackberry SDK

  7. Click Next and Install

  8. JRE Configuration

      Go Window -> Preferences -> Java -> Installed JRE

9. Click Add and Select Blackberry Execution Environment VM & Click next

 

10. Add Definition file. Browse and select Blackberry.ee file from installed path

       <path>/eclipse/Plugins/net.rim.ejde.componentpack.7.0/components/BlackBerry.ee

11. Click Ok and Finish

12. Use it.

Other Helpful (Optional) Tools on Development Environment

1. Mobile File Manager : from http://appworld.blackberry.com/webstore/content/85979/?lang=EN

This awesome program allows you to view and manage the file system on the BlackBerry, edit files and even setup an FTP from the BlackBerry to a server. I did not use the FTP feature.

2 . BlackBerry Desktop Manager :

from http://blackberry-desktop-software.en.softonic.com/blackberry. Some variant of this program comes with the BlackBerry when you buy it. But if you don’t have one, you can get it here. You need this to install programs to a physical BlackBerry. I have a variant from Roxio that also allows me to transfer files between the BlackBerry and the PC.



                                                                                         -PAVANKUMAR JOSHI

SPHINX SEARCH ENGINE INSTALLATION

1. Installing Sphinx on Linux platform.
Required Tools
i) a working C++ compiler. GNU gcc is known to work.
ii) a good make program. GNU make is known to work.

Installation Steps:
1. Download Sphinx source tarball from
http://sphinxsearch.com/downloads/release/
download sphinx-2.0.4-release.tar.gz.
2.
Extract everything from the distribution tarball (haven't you already?) and
go to the sphinx sub directory. (We are using version 2.0.1-beta here for
the sake of example only; be sure to change this to a specific version you're using.

$ tar -xzvf sphinx-2.0.4-release.tar.gz
$cd sphinx.

3.

Run the configuration program:
$ ./configure
There's a number of options to configure. The complete listing may be obtained by using --help switch. The most important ones are:
i) --prefix, which specifies where to install Sphinx;
such as --prefix=/usr/local/sphinx (all of the examples use this prefix)
ii) --with-mysql, which specifies where to look for MySQL include
and library files, if auto-detection fails;
iii) --with-pgsql, which specifies where to look for PostgreSQL include
and library files.

4. Build the binaries:

$ make

5. Install the binaries in the directory of your choice:
   (defaults to /usr/local/bin on *nix systems, but is overridden with configure -prefix).
   $ make install
   

2. Installing Sphinx on Window platform.
Required tools:
On Windows, you will need Microsoft Visual C/C++ Studio .NET 2005 or above. Other compilers/environments will probably work as well, but for the time being, you will have to build makefile (or other environment specific project files) manually. 
 

Installation Steps:

Download Sphinx source tarball from
http://sphinxsearch.com/downloads/release/
download sphinx-2.0.4-win32.zip or sphinx-2.0.4-win32-pgsql.zip.

1. Extract everything from the .zip file you have downloaded – sphinx-2.0.4-win32.zip or sphinx-2.0.4-win32-pgsql.zip if you need PostgresSQL support as well. You can use Windows Explorer in Windows XP and up to extract the files, or a freeware package like 7Zip to open the archive.
   For the remainder of this guide, we will assume that the folders are unzipped into c:\sphinx, such that searchd.exe can be found in c:\sphinx\bin\searchd.exe . If you decide to use any different location for the folders or configuration file, please change it accordingly.
   
   
2. Edit the contents of sphinx.conf.in - specifically entries relating to @CONFDIR@ - to paths suitable for your system.
   
   
3. Install the searchd system as a Windows service:
   C:\Sphinx\bin> C:\Sphinx\bin\searchd --install –config
C:\Sphinx\sphinx.conf.in --servicename SphinxSearch
   
   
4. The searchd service will now be listed in the Services panel within the Management Console, available from Administrative Tools. It will not have been started, as you will need to configure it and build your indexes with indexer before starting the service.
   

Known Installation issues

1. Standard set of installation (PostgreSQL Issues)

./configure

make

make install.

Default supports Mysql database.

Solution: If use PostgreSQL Use

./configure –with-pgsql=/usr/local/include/postgresql

make

make install. 

2. If configure fails to locate MySQL headers and/or libraries,

Solution: try checking for and installing mysql-devel package.
On some systems, it is not installed by default.   

3. If make fails with a message which look like

          /bin/sh: g++: command not found
          make[1]: *** [libsphinx_a-sphinx.o] Error 127

Solution: try checking for and installing gcc-c++ package.

4. If you are getting compile-time errors which look like

         sphinx.cpp:67: error: invalid application of `sizeof' to
          incomplete type `Private::SizeError<false>'
         Solution:   this means that some compile-time type size check failed. 

                           The most probable   reason is that off_t type is less than 64-bit on your system. 

                         As a quick hack, you can edit sphinx.h and replace off_t with DWORD in a typedef for SphOffset_t,

     but note that this will prohibit you from using full-text indexes larger than 2 GB. Even if the hack helps, please report such issues,

 providing the exact error message and compiler/OS details, so I could properly fix them in next releases.  

5. While doing a make on
 sphinx we got:

sphinx.cpp:16792: error: ‘m_pMysqlFields’ was not declared in this scope
sphinx.cpp:16793: error: ‘m_pMysqlResult’ was not declared in this scope
sphinx.cpp:16793: error: ‘mysql_fetch_fields’ was not declared in this scope
sphinx.cpp:16795: error: ‘m_pMysqlFields’ was not declared in this scope
make[2]: *** [sphinx.o] Error 1
make[2]: Leaving directory `/usr/local/src/sphinx-0.9.8.1/src'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/local/src/sphinx-0.9.8.1/src'
make: *** [all-recursive] Error 1
Solution: simply install mysql-devel rpm and should compile fine.

6. While doing a ./confuger on sphinx we got:

         sphinx.h:54:19: error: mysql.h: No such file or directory.

       Solution: simply install mysql-devel rpm and should compile fine.

                                                                                   -PAVANKUMAR JOSHI

SPHINX SEARCH ENGINE BASIC DETAILS

         Sphinx is a full-text search engine that's fast, relevant, has native support for indexing SQL databases, can talk its own SQL dialect, and got a bunch of other advanced features.

License :  publicly distributed under GPL version 2 &
                Commercial licensing (eg. for embedded use) is available upon request.

Access Methods: 
                     Applications can access Sphinx search daemon (searchd) using any of the three different access methods:
        a) via native search API (SphinxAPI),
        b) via Sphinx own implementation of MySQL network protocol (using a small SQL subset called SphinxQL), or
        c) via MySQL server with a pluggable storage engine (SphinxSE).

Supported Databases : MySQL, PostgreSQL, MSSQL, Oracle.

Supported Languages: Java, PHP, Perl ,Ruby, C  and Python.

Supported Operating systems:
        i.   Linux 2.4.x, 2.6.x (many various distributions)
        ii.  Windows 2000, XP
        iii. FreeBSD 4.x, 5.x, 6.x, 7.x
        iv   NetBSD 1.6, 3.0
         v   Solaris 9, 11
        vi   Mac OS X

(For detailed description go through http://sphinxsearch.com/docs/2.0.4/ )             

                                  ADVANTAGES (KEY FEATURES)
           
    Key Sphinx features are:
1. high indexing and searching performance.
2. advanced indexing and querying tools (flexible and feature-rich text t  
    tokenizer, querying language, several different ranking modes, etc).
3. advanced result set post-processing (SELECT with expressions, WHERE, ORDER BY, GROUP BY etc over text search results).
4. proven scalability up to billions of documents, terabytes of data, and thousands of queries per second.
5. easy integration with SQL and XML data sources, and SphinxAPI, SphinxQL, or SphinxSE search interfaces.
6. easy scaling with distributed searches.
7. has high indexing speed (upto 10-15 MB/sec per core on an internal benchmark).
8. has high search speed (upto 150-250 queries/sec per core against 1,000,000 documents, 1.2 GB of data on an internal benchmark).
9. has high scalability (biggest known cluster indexes over 3,000,000,000 documents, and busiest one peaks over 50,000,000 queries/day).
10.provides good relevance ranking through combination of phrase proximity ranking and statistical (BM25) ranking.
11.provides distributed searching capabilities.
12.provides document excerpts (snippets) generation.
13.provides searching from within application with SphinxAPI or SphinxQL interfaces, and from within MySQL with pluggable SphinxSE storage engine.
14.supports boolean, phrase, word proximity and other types of queries.
15.supports multiple full-text fields per document (upto 32 by default).
16.supports multiple additional attributes per document (ie. groups, timestamps, etc).
17.supports stopwords, morphological word forms dictionaries, tokenizing exceptions,  both single-byte encodings and UTF-8
18.supports PostgreSQL, ODBC compliant databases (MS SQL, Oracle, etc), MySQL natively;


                                      OTHER SEARCH ENGINES
   LUCENE:
    Lucene is a full-text search engine and open source .
    Features:
             a. Scalable and high performance indexing (Incremental indexing and
                 small RAM requirement- 1MB).
             b. Powerful, Accurate and Efficient search Algorithms.
                  ( Ranking mechanism, query mechanism,multiple indexing,
                    searching while update/add ).
            
             License :  Apache license(Open source).
             Supported Langauges: Java, PHP, Perl, Ruby, Lisp, Delpi, C, C++,.NET .
             Supported Documents : XML, HTML, Openoffice documents,
                                                   Word documents.
            
                                       SPHINX vs LUCENE
              1. Lucene beats Sphinx on features.
              2. Sphinx beats lucene on performance,indexing and querying.
              3. Sphinx supports real time indexing, database indexing, which is
                  not directly supported by lucene.
               4. sphnix indexes occupy more space as compared to luence indexes.
                   But as sphinix is  real time search engine library so it is fast as compared
                   to lucene.

      2. SPHIDER:

    Sphider is a full-text search engine and open source . Its language dependent
     this is built for PHP.



                                                                                         -PAVANKUMAR JOSHI