PHP Classes

Searchy PHP Search Engine: Crawl, index and search multiple Web sites

Recommend this page to a friend!
  Info   View files Example   View files View files (13)   DownloadInstall with Composer Download .zip   Reputation   Support forum (1)   Blog (1)    
Last Updated Ratings Unique User Downloads Download Rankings
2023-05-18 (7 months ago) RSS 2.0 feedNot enough user ratingsTotal: 158 This week: 2All time: 8,963 This week: 204Up
Version License PHP version Categories
searchy 1.0Custom (specified...5Searching, Applications
Description 

Author

This package can crawl, index, and search multiple Web sites.

It provides a simple application to let users add sites to be crawled.

The package can crawl the pages of the given sites, parse the page contents and add the contents to a database that index the pages by body contents and title.

It can also let users search for content that was previously indexed pages using given keywords and displays the top results.

Innovation Award
PHP Programming Innovation award nominee
May 2023
Number 6
Popular search engines like Google are handy for finding content on sites where users don't know where those sites are.

When users want to search more specific sites that they know, general search engines like Google are not ideal.

This package provides a PHP application that lets users index and search only the sites they want.

Manuel Lemos
Picture of Faris AL-Otabi
  Performance   Level  
Name: Faris AL-Otabi <contact>
Classes: 12 packages by
Country: Saudi Arabia Saudi Arabia
Innovation award
Innovation award
Nominee: 3x

Recommendations

What is the best PHP website crawler class?
Download a complete website to local storage

Example

<?php

include_once 'config.php';
include_once
'src/Database.php';
include_once
'src/SearchEngine.php';

if (!isset(
$_GET['k'])) {
   
header('Location: index.php');
}

$keyword = $_GET['k'];

$page = (!isset($_GET['page'])) ? 1 : $_GET['page'];

$db = new Database($config);
$se = new SearchEngine($db);

$sites = $se->findSites($keyword, $page);

?>

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Searchy - <?php echo $keyword; ?></title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet">
</head>

<body>
    <nav class="navbar navbar-expand-lg navbar-dark bg-dark">
        <div class="container-fluid">
            <a class="navbar-brand" href="#">Searchy</a>
            <button class="navbar-toggler" type="button" data-bs-toggle="collapse" data-bs-target="#navbarColor02" aria-controls="navbarColor02" aria-expanded="false" aria-label="Toggle navigation">
                <span class="navbar-toggler-icon"></span>
            </button>

            <div class="collapse navbar-collapse" id="navbarColor02">
                <ul class="navbar-nav me-auto">
                    <li class="nav-item">
                        <a class="nav-link" href="index.php">Home</a>
                    </li>
                    <li class="nav-item">
                        <a class="nav-link" href="add.php">Add Site</a>
                    </li>
                    <li class="nav-item">
                        <a class="nav-link" href="crawl.php">Crawl Site</a>
                    </li>
                </ul>
            </div>
        </div>
    </nav>

    <div class="container">
        <div class="pt-3">
            <form class="d-flex" method="GET" action="search.php">
                <input class="form-control me-sm-2" type="text" name="k" value="<?= $_GET['k']; ?>" placeholder="Search">
                <button class="btn btn-secondary my-2 my-sm-0" type="submit">Search</button>
            </form>
        </div>

        <div class="pt-4">
            <div class="d-flex justify-content-between">
                <p>You searched for <b>"<?php echo $keyword ?>"</b></p>

                <p><b><?php echo $sites['count']; ?></b> results found</p>
            </div>
        </div>

        <hr />

        <div class="pt-3">
            <ul>
                <?php foreach ($sites['results'] as $site) : ?>

                    <li>
                        <a href="<?php echo $site->url ?>">
                            <h3><?php echo $site->title ?></h3>
                        </a>
                        <p><?php echo $site->blurb ?></p>
                    </li>

                <?php endforeach; ?>
</ul>
        </div>

        <div class="pt-5">
            <ul class="pagination pagination-sm">
                <?php for ($page = 1; $page <= $sites['pages']; $page++) : ?>
<li class="page-item <?php echo isset($_GET['page']) && $_GET['page'] == $page ? 'active' : '' ?>">
                        <a class="page-link" href="search.php?k=<?= $keyword ?>&page=<?= $page ?>"><?= $page; ?></a>
                    </li>
                <?php endfor; ?>
</ul>
        </div>
    </div>


    <script src="https://cdn.jsdelivr.net/npm/@popperjs/core@2.9.2/dist/umd/popper.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/js/bootstrap.min.js"></script>
</body>

</html>


Details

Searchy

PHP Search Engine

Features

  1. Basic Search Engine Features
  2. Simple Crawler
  3. Add Site Feature

How to use

  1. `$ composer update`
  2. Update crawler/urls.txt with the urls you want to index like Alexa top 1000
  3. Update config.php
  4. Import search.sql to your database
  5. Run `$ php crawler.php `
  6. Enjoy

Screenshots

$ php crawler.php
Indexing https://www.google.com/
Indexing https://www.youtube.com/
Indexing https://www.facebook.com/
Indexing https://www.baidu.com/
Indexing https://www.wikipedia.org/
Indexing https://twitter.com/
Indexing https://www.yahoo.com/
Indexing https://www.instagram.com/
Indexing Finished

![](https://i.imgur.com/M5ZMCcJ.png)

![](https://i.imgur.com/AS0b4VT.png)

![](https://i.imgur.com/DyPSiwU.png)

License

MIT


  Files folder image Files  
File Role Description
Files folder imagecrawler (2 files)
Files folder imagesrc (2 files)
Accessible without login Plain text file add.php Example Example script
Accessible without login Plain text file composer.json Data Auxiliary data
Accessible without login Plain text file config.php Aux. Auxiliary script
Accessible without login Plain text file crawl.php Example Example script
Accessible without login Plain text file index.php Aux. Auxiliary script
Accessible without login Plain text file LICENSE Lic. License text
Accessible without login Plain text file README.md Doc. Documentation
Accessible without login Plain text file search.php Example Example script
Accessible without login Plain text file search.sql Data Auxiliary data

  Files folder image Files  /  crawler  
File Role Description
  Accessible without login Plain text file crawler.php Example Example script
  Accessible without login Plain text file urls.txt Data Site URLs file list

  Files folder image Files  /  src  
File Role Description
  Plain text file Database.php Class Class source
  Plain text file SearchEngine.php Class Class source

 Version Control Unique User Downloads Download Rankings  
 100%
Total:158
This week:2
All time:8,963
This week:204Up