Anton Babenko

Splunk Enterprise on Amazon Linux

May 19, 2012 Web-development Leave a comment

Splunk is one of the best tool for log analysis with some flaws in Free edition (no user authentication and 500mb indexing limit per day). Most setups don’t need to have user authentication, because splunk can stay behind nginx proxy and have http basic auth there. Also for many sites the limit of 500mb/day is fairy high limit. Though, one of my site has been indexing more than 500mb of logs per day several times per month and according to Splunk Free License it broke the terms of use (huh?), which means that I couldn’t even search on already indexed logs.

Splunk Enterprise Trial version works for 60 days and then you have to migrate to Free license (see above) or buy Enterprise license (it is very expensive if you want to have it on small sites like my imagepush.to). The solution I found is to reinstall it every 2 months:


# remove splunk, if you have it installed already
sudo rpm -e splunk

# Splunk for Amazon Linux (32bit)
curl -O http://216.221.226.44/releases/4.3.2/splunk/linux/splunk-4.3.2-123586.i386.rpm
sudo rpm -U splunk-4.3.2-123586.i386.rpm
sudo /opt/splunk/bin/splunk start


Then login to splunk web and set password. All settings will be in place already.

How to use Zend_Feed in Symfony2 project

Mar 24, 2012 Web-development Leave a comment

There are several Symfony2 bundles to generate feeds (FeedBundle, for example). For imagepush.to I have to generate a variety of feeds and none of available bundles was good enough, so I want to use Zend_Feed. I didn’t want to clone bloated Zend Framework into my project, and luckily KnpLabs have made repos with all popular Zend components already. Here is what I have in deps:

[zend-feed]
    git=http://github.com/KnpLabs/zend-feed.git
    target=Zend/Feed

[zend-loader]
    git=http://github.com/KnpLabs/zend-loader.git
    target=Zend/Loader

[zend-stdlib]
    git=http://github.com/KnpLabs/zend-stdlib.git
    target=Zend/Stdlib

and this is in autoload.php:

$loader->registerNamespaces(array(
    // ...
    'Zend'            => __DIR__.'/../vendor'
    // ...
));

Elastica: reindexing on demand

Mar 9, 2012 Web-development Leave a comment

I used to make sure that Elastica is updating search index after record is created or updated, and there are listeners as specified in documentation. It works fine for basic needs out of the box, but I have some index fields, which were populated from another related objects, which I do lazy-fetch, so I ended up with this code:

// do something with $property
$em->persist($property);
$em->flush();
$em->clear();

// reload property from the database to get saved data with all related objects needed for index
$property = $em->find('MyBundle:Property', $property->getId());

// reindex
$type = $this->kernel->getContainer()->get('foq_elastica.index.website.property');
$modelTransformer = new \FOQ\ElasticaBundle\Transformer\ModelToElasticaAutoTransformer();
$mapping = $type->getMapping();
$fields = array_keys($mapping["property"]["properties"]);

$objectPersister = new \FOQ\ElasticaBundle\Persister\ObjectPersister($type, $modelTransformer, 'My\MyBundle\Entity\Property', $fields);
$objectPersister->replaceOne($property);

The trick here is to clear doctrine entity manager and then fetch object to replace one more time.

Easy text search using Elastic Search

Jan 25, 2012 Web-development Leave a comment

Recently I used to add very plain text search to ne.no and since I’m using Elastic Search with ElasticaBundle it was easy to achieve.

$textQuery = new \Elastica_Query_QueryString();
$textQuery->setFields(array("address", "name", "city", "searchPartNames", "searchCompanyName"));
$textQuery->setDefaultOperator("AND");
$textQuery->setQueryString($selectedText);

$tmpTexts = array();
foreach (explode(" ", $selectedText) as $oneWord) {
    $oneWord = trim($oneWord);
    if ($oneWord == "")
        continue;
    if (preg_match("/^[0-9]+/", $oneWord)) {
        $tmpTexts[] = $oneWord;
    } else {
        $tmpTexts[] = $oneWord . "*";
    }
}
$selectedText = implode(" ", $tmpTexts);

Added star-wildcard after all words, but not after numbers, so that user is able to find correct building address located on Drammensgt or Drammensvn if he searches for “Drammens 171″.

Lego

Jan 23, 2012 Fun Leave a comment

Ne.no has been released

Jan 15, 2012 Web-development Leave a comment

Ne.no is out and working well. It is my second project built using Symfony2 framework.

This blog v2

Jan 15, 2012 Life Leave a comment

After several years of silence I decided to continue blogging about things I read, see or just want to mention.