dotcmis / alfresco / delete => create => search delay

Question

I try to integrate dotcmis and alfresco into my application. When creating my unit tests, I face this issue : - I set up my test environment by deleting "myfolder" if any - I create back myfolder and put a document into it

then I try to find the document : - The first time (when myfolder does not exist before), Search returns 0 results - Next times, when myfolder exists before and is deleted by my test setup, I get an exception :

Apache Chemistry OpenCMIS - runtime error
HTTP Status 500 - <!--exception-->runtime<!--/exception--><p><!--message-->Node does     not exist: missing://missing/missing(null)<!--/message--></p><hr noshade='noshade'/><!--    stacktrace--><pre>
org.apache.chemistry.opencmis.commons.exceptions.CmisRuntimeException: Node does not exist: missing://missing/missing(null)
at org.alfresco.opencmis.AlfrescoCmisExceptionInterceptor.invoke(AlfrescoCmisExceptionInterceptor.java:80)
at ...

When I go to Alfresco, the document exists. It seems that the folder and document are not yet disponible for query, but why ? If I put in comment the test environment init, the document is found

Maybe I do something wrong but what ?

Here is my code :

[TestMethod()]
[DeploymentItem(@"Files\SearchTest_1", @"Files\SearchTest_1")]
public void SearchTest_2()
{
    string myfoldername = "myfolder";

    // Session creation
    var p = new Dictionary<String, String>();
    p[SessionParameter.User] = _userName;
    p[SessionParameter.Password] = _userPassword;
    p[SessionParameter.BindingType] = BindingType.AtomPub;
    p[SessionParameter.AtomPubUrl] = _serverUrl;

    var session = DotCMIS.Client.Impl.SessionFactory.NewInstance().GetRepositories(p)[0].CreateSession();
    session.DefaultContext.CacheEnabled = false;
    var operationContext = session.CreateOperationContext();
    operationContext.IncludeAcls = true;

    // Delete and create back folder and document
    // /*
    DotCMIS.Client.IFolder rootFolder = this._testSession.GetRootFolder(operationContext);
    DotCMIS.Client.IFolder myFolder = null;
    Dictionary<String, Object> properties = null;

    // Le dossier de destination des tests existe-t-il ?
    var myFolderExists = rootFolder.GetChildren(operationContext).Any(child => child.Name.Equals(myfoldername));
    if (myFolderExists)
    {
        myFolder = (DotCMIS.Client.IFolder)session.GetObjectByPath(String.Format(@"/{0}", myfoldername), operationContext);
        myFolder.DeleteTree(true, DotCMIS.Enums.UnfileObject.Delete, true);
    }

    properties = new Dictionary<String, Object>();
    properties[PropertyIds.Name] = myfoldername;
    properties[PropertyIds.ObjectTypeId] = "cmis:folder";
    myFolder = rootFolder.CreateFolder(properties);
    rootFolder.Refresh();

    myFolder = (DotCMIS.Client.IFolder)session.GetObjectByPath(String.Format(@"/{0}", myfoldername), operationContext);

    FileInfo sourceFile = new FileInfo(@"Files\SearchTest_1\SearchTest_1.pdf");
    properties = new Dictionary<String, Object>();

    properties[PropertyIds.ObjectTypeId] = "cmis:document";
    properties[PropertyIds.Name] = sourceFile.Name;

    using (var fileStream = sourceFile.OpenRead())
    {
        var contentStream = new DotCMIS.Data.Impl.ContentStream();
        contentStream.MimeType = "application/pdf";
        contentStream.Stream = fileStream;
        contentStream.Length = fileStream.Length;
        //this._testSession.CreateDocument(properties, this._testSession.CreateObjectId(myFolder.Id), contentStream, null);
        DotCMIS.Client.IDocument createdDocument = myFolder.CreateDocument(properties, contentStream, null);
    }

    // */

    // Recherche
    string query = @"SELECT * FROM cmis:document WHERE cmis:name = 'SearchTest_1.pdf'";
    var results = this._testSession.Query(query, false, operationContext).ToArray();
    Assert.AreEqual(1, results.Length);
}

Answer 1

I don't think the "eventually consistent" aspect of solr is an issue in production. It may be when testing, but I prefer having a better system in production and having some problems in debug than the other way round.

To solve my issue in debug I first put a Thread.Sleep(20)... and it worked, but.. it's quite long while debugging.

My second solution that seem to work is to check the indexing state of solr using the url address:8080/solr/admin/cores?action=REPORT (_solrStateUrl in my code). The problem is that it's only accessible by SSL by default. So you need to get the file "browser.p12" from the afresco server and put it in your project.

So I made 2 methods : - CheckIndexingState that is parsing the xml response to find transactions in progress - WaitForIndexingDone that is looping on CheckIndexingState

This code may not be very safe, but it's just for testing...

Here it is. Hope this will help someone...

private bool CheckIndexingState()
{
    ServicePointManager.ServerCertificateValidationCallback += new RemoteCertificateValidationCallback(delegate(object sender2, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; });
    System.Net.HttpWebRequest request = (HttpWebRequest)System.Net.WebRequest.Create(_solrStateUrl);
    request.ClientCertificates.Add(new X509Certificate2(@"ssl/browser.p12", "alfresco"));
    var sb = new System.Text.StringBuilder();
    byte[] buffer = new byte[256];
    int nbRead = -1;
    using (var stream = request.GetResponse().GetResponseStream())
    {
        while (nbRead != 0)
        {
            nbRead = stream.Read(buffer, 0, 256);
            sb.Append(System.Text.Encoding.UTF8.GetString(buffer, 0, nbRead));
        }
    }

    String state = sb.ToString();

    try
    {
        XmlDocument doc = new XmlDocument();
        doc.LoadXml(sb.ToString());

        var node = doc.SelectSingleNode(@"/response/lst[@name='report']/lst[@name='alfresco']/long[@name='Count of transactions in the index but not the DB']");
        int count = Int32.Parse(node.InnerText);
        if (count > 0)
            return false;

        node = doc.SelectSingleNode(@"/response/lst[@name='report']/lst[@name='alfresco']/long[@name='Count of acl transactions in the index but not the DB']");
        count = Int32.Parse(node.InnerText);
        if (count > 0)
            return false;

        node = doc.SelectSingleNode(@"/response/lst[@name='report']/lst[@name='alfresco']/long[@name='Count of missing transactions from the Index']");
        count = Int32.Parse(node.InnerText);
        if (count > 0)
            return false;

        node = doc.SelectSingleNode(@"/response/lst[@name='report']/lst[@name='alfresco']/long[@name='Count of missing acl transactions from the Index']");
        count = Int32.Parse(node.InnerText);
        if (count > 0)
            return false;
    }
    catch (Exception)
    {
        throw;
    }

    return true;

}

Answer 2

Are you using Alfresco 4.0 with Solr for indexing? The Solr index is eventually consistent, this means that it can take a while (up to 15 seconds in the default configuration) for updates to show up in search results.

If you need updates to show up immediately you could switch to Lucene as the indexing subsystem.