How to develop Compass-like functionality for Apache Solr

In my entry on how to get Compass to work with EclipseLink 2.2, I described the changes needed to get these two great pieces of software to work together. Since I am always evaluating different approaches, components and because it seems that Compass development for the current version 2.x is in maintenance mode I decided to evaluate Apache Solr. Scouring the web in my favorite search engine, I did not find a Compass-like alternative for Apache Solr and whether Compass actually supports Apache Solr I am not sure.

Apache Solr supports a variety of languages and it able to integrate with a host of 3rd party applications but I was unable to find a framework or a component similar to Compass to integrate the Object Relation Mapping (for example EclipseLink) performed in the domain layer of a business application with the search capabilities of Apache Solr.

Working with Compass, I found the annotation of the domain class as a natural fit and so I decided to follow the same idea with Solarflare. The domain class is annotated first with the @Searchable to indicate to the framework that the class should be considered for indexing.

<pre>@Target({TYPE}) @Retention(RUNTIME) @Documented @Qualifier
public @interface Searchable {
    String name();
}</pre>

Apache Solr has the requirement that an indexed document has a unique id, therefore for a domain object instance to be indexed by Solarflare the class must also be annotated with the @SearchablePropertyId. Currently the implementation ignores the boost value of the annotation.

<pre>@Target({FIELD}) @Retention(RUNTIME) @Documented @Qualifier
public @interface SearchablePropertyId {
    /** The name of the property / field.
     * If the default is used then the name is derived from the field / property name
     */
    String name() default "";

    /** The boost that should be applied */
    float boost() default 0;
}</pre>

Of course for the object to be found when searched, the class needs to be annotated with @SearchableProperty to defined the fields that should be added to the Apache Solr document. The field can be annotated with a specific SearchablePropertyType but in most cases the SearchablePropertyType.INFER can be used.

@Target({FIELD}) @Retention(RUNTIME) @Documented @Qualifier
public @interface SearchableProperty {
    /** The name of the searchable property */
    String name() default "";

    /** The boost factor of the property */
    float boost() default 0;

    /** Whether the field should be mapped to a dynamic field in the search provider */
    boolean dynamic() default false;

    /** The type of the dynamic field */
    SearchablePropertyType type() default SearchablePropertyType.INFER;
}

So this is all fine and well, but how does a class look like that is annotated to be indexed by Apache Solr? Good question and below you’ll find the answer which is taken from the unit test with Solarflare. The name class defines the name of a person, i.e. first name and last name. Since we do not want Solrflare to index this class standalone, the @Searchable and @SearchablePropertyId annotations are not used.

public class Name implements Serializable {
    private static final long serialVersionUID = 1L;

    @SearchableProperty(name = "name")
    private String firstName;

    @SearchableProperty(dynamic = true)
    private String lastName;

    /**
     * Default constructor
     */
    public Name() {}

    /**
     * Constructs the new name
     * @param firstName The first name
     * @param lastName The last name
     */
    public Name(String firstName, String lastName) {
        this.firstName = firstName;
        this.lastName = lastName;
    }

    /**
     * Returns the first name
     * @return String
     */
    public String getFirstName() {
        return firstName;
    }

    /**
     * Sets the first name
     * @param firstName The new value
     */
    public void setFirstName(String firstName) {
        this.firstName = firstName;
    }

    /**
     * Returns the last name
     * @return String
     */
    public String getLastName() {
        return lastName;
    }

    /**
     * Sets the last name
     * @param lastName The new value
     */
    public void setLastName(String lastName) {
        this.lastName = lastName;
    }

    /**
     * Returns a string representation of the object
     * @return String
     */
    @Override
    public String toString() {
        return "Name{" +
                "firstName='" + firstName + '\'' +
                ", lastName='" + lastName + '\'' +
                '}';
    }

Next is the Person class which defines the date of birth field and declares a field of type Name. Since Person is the class that we would like to index and later search for, it is annotated with the @Searchable and @SearchablePropertyId.

@Searchable(name = "Person")
public class Person implements Serializable {
    private static final long serialVersionUID = 1L;

    @SearchablePropertyId
    private Long id;

    @SearchableProperty
    private Name name;

    @SearchableProperty(name = "birthDate", dynamic = true)
    private Date dateOfBirth;

    /**
     * Returns the object identity
     * @return Long
     */
    public Long getId() {
        return id;
    }

    /**
     * Sets the object identity of the entity
     * @param id The new value
     */
    public void setId(Long id) {
        this.id = id;
    }

    /**
     * Returns the name
     * @return String
     */
    public Name getName() {
        return name;
    }

    /**
     * Sets the name
     * @param name The new value
     */
    public void setName(Name name) {
        this.name = name;
    }

    /**
     * Returns the date of birth
     * @return Date
     */
    public Date getDateOfBirth() {
        return dateOfBirth;
    }

    /**
     * Sets the birth date
     * @param dateOfBirth The new value
     */
    public void setDateOfBirth(Date dateOfBirth) {
        this.dateOfBirth = dateOfBirth;
    }

    /**
     * Returns a string representation of the object
     * @return String
     */
    @Override
    public String toString() {
        return "Person{" +
                "id=" + id +
                ", name=" + name +
                ", dob=" + dateOfBirth +
                '}';
    }
}

And here is the unit test that demonstrates how to index a domain object and then to query for it.

    /**
     * Tests indexing a single entity object by calling the SearchManager directly
     * @throws Exception If there was an exception
     */
    @Test
    public void testIndexObject() throws Exception {

        Person person = new Person();
        person.setId(900L);
        person.setName(new Name("Mark", "Ashworth"));
        person.setDateOfBirth(new SimpleDateFormat("yyyy-MM-dd").parse("1974-05-31"));

        SearchManager search = new SearchManagerBean(new EmbeddedSolrServer(h.getCoreContainer(), h.getCore().getName()));
        search.onSearchableAdded(person);

        List<Object> result = search.search(Person.class, "Mark");
        assertEquals(result.size(), 1);
        Person o = (Person) result.get(0);

        assertEquals("The id is not the same", new Long(900L), o.getId());
        assertEquals("The name.firstName is not the same", "Mark", o.getName().getFirstName());
        assertEquals("The name.lastName is not the same", "Ashworth", o.getName().getLastName());
    }

Now you are saying that is all fine and well, but you said that this puppy would integrate with the Java Persistence API and that I did not need to worry about calling the SearchManager directly. Yes I did, and patience my padawan first the foundation lay, we must. The integration comes from the SearchIndexListener which is triggered for specific events during the object life cycle to either index it or to delete it from the index.

public class SearchIndexListener {

    private SearchManagerBean search = new SearchManagerBean();

    /**
     * Triggered after a JPA entity is persisted or updated
     * @param entity The entity to index
     */
    @PostPersist
    @PostUpdate
    public void postPersist(Object entity) {
        search.onSearchableAdded(entity);
    }

    /**
     * Triggered after a JPA entity is deleted
     * @param entity The entity to remove from the index
     */
    @PostRemove
    public void postRemove(Object entity) {
        search.onSearchableRemoved(entity);
    }

All that remains is to instruct EclipseLink to call the listener and this is done in the eclipselink-orm.xml.

<entity-mappings
	xmlns="http://www.eclipse.org/eclipselink/xsds/persistence/orm"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.eclipse.org/eclipselink/xsds/persistence/orm http://www.eclipse.org/eclipselink/xsds/eclipselink_orm_2_3.xsd"
	version="2.3">

    <persistence-unit-metadata>
        <persistence-unit-defaults>
            <entity-listeners>
                <entity-listener class="za.co.yellowfire.solarflare.jpa.SearchIndexListener">
                    <post-persist method-name="postPersist"  />
                    <post-remove method-name="postRemove"  />
                </entity-listener>
                <entity-listener class="za.co.yellowfire.jpa.DomainEntityListener">
                    <pre-persist method-name="prePersist"  />
                    <pre-update method-name="preUpdate"  />
                </entity-listener>
            </entity-listeners>
        </persistence-unit-defaults>
    </persistence-unit-metadata>
</entity-mappings>

Of course Apache must be running in a JEE server, for example Glassfish. Solarflare currently reads the Glassfish system property yellowfire.solarflare.url to determine web address of the Apache Solr instance that documents should be indexed to and queried from.

@Named("SearchProvider")
public class SearchProvider {
    private static final Logger LOGGER = LoggerFactory.getLogger(LogType.PROVIDER.getCategory());

    /**
     * Provides the Solr URL that should be used
     * @return The Solr URL
     */
    @Produces @Solr @Url
    public String provideSearchUrl() {
        String url = System.getProperty("yellowfire.solr.url", "http://localhost:8080/solr");

        if (LOGGER.isDebugEnabled()) {
            LOGGER.debug("Providing Solr url {}", url);
        }
        return url;
    }

And that dear friends is the introduction to creating a Compass-like framework to expose the domain classes to Apache Solr. If you have suggestions or comments on how to make Solarflare even better then feel free to swing past the project page at http://www.ohloh.net/p/solar-flare or get the source at http://code.google.com/p/solar-flare/.

About these ads

Tags:

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Java Magic

Blog about Tapestry5, Plastic and related technologies

Steps & Leaps

Thoughts and Comments on (Mainly) Creativity, Innovation and Management

Facilitating Agility

Scrum and Agile Facilitation

Jan's Blog

Mainly development and technology stuff I haven't easily found in the net

WatirMelon

A 93% Software Testing Blog by Alister Scott

Dan Haywood

domain driven design, restful objects, apache isis, the naked objects pattern, agile and more

Marko A. Rodriguez

Supporting the Emerging Graph Landscape

A developer's journal

On Oracle, JEE, SOA and whatmore

RedStack

Musings on Integration with Oracle Fusion Middleware

oracle-stack-support

Oracle Stack Support (One Window Support)

Struberg's Blog

Yet another blog site?

Exit Condition

Andrew Lee Rubinger

Antonio's Blog

A blog mainly about Java

Sematext Blog

Search, Big Data, Analytics, Natural Language Processing

WordPress.com News

The latest news on WordPress.com and the WordPress community.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: