Azure Storage Tables - Not all properties from TableEntity are being stored after InsertOperation

Question

So the problem is: I have an entity which has something like 40 properties (all properly defined in code as "public String PropertyName {get;set;}". When I insert new entities majority of properties are being stored, but some of them are not.

The code is following:

public class PostTableEntity : TableEntity
{
    #region Fields
    #endregion

    #region Properties
    public Guid CreatorId { get; set; }
    public String HtmlText { get; set; }
    public String SubjectIds { get; set; }
    public String QuoteString { get; set; }
    public double GeoLat { get; set; }
    public double GeoLong { get; set; }
    public String GeoPlace { get; set; }
    public Int32 TotalSmiles { get; set; }
    public DateTime DateUTC { get; set; }
    public Guid? EventId { get; set; }
    public string EventName { get; set; }
    public String ExcludedUsers { get; set; }
    public String Comment00_Text { get; set; }
    public Guid Comment00_UserId { get; set; }
    public Guid Comment00_CommentId { get; set; }
    {...} //Some more props - no more than 30 in total        
    public String VeryImportantData { get; set; }
    #endregion

    #region Constructors
    public PostTableEntity()
    {

    }

    public PostTableEntity(String partitionKey, String rowKey, Guid creatorId, DateTime dateUTC, String htmlText)
        : base(partitionKey, rowKey)
    {
        this.CreatorId = creatorId;
        this.HtmlText = htmlText;
        this.DateUTC = dateUTC;
    }
    #endregion

    #region Methods
    public void SetSubjectIdsList(List<Guid> subjectIds)
    {
        if (subjectIds != null)
        {
            this.SubjectIds = String.Join(";", subjectIds);
        }
        else
        {
            this.SubjectIds = "";
        }
    }
    #endregion
}

... then there's a deriving class:

public class ImagePostTableEntity : PostTableEntity
{
    #region Fields
    #endregion

    #region Properties
    public String StorageAccountName { get; set; }
    public String StorageContainerName { get; set; }
    public String BlobName_Original { get; set; }
    public String BlobName_Large { get; set; }
    public String BlobName_Medium { get; set; }
    public String BlobName_Small { get; set; }      
    #endregion

    #region Constructors
    public ImagePostTableEntity()
    {

    }

    public ImagePostTableEntity(String partitionKey, String rowKey, Guid creatorId, DateTime date, String htmlText, List<Guid> subjectIds, String storageAccountName, String storageContainerName, String blobName_Original, String blobName_Large, String blobName_Medium, String blobName_Small)
        : base(partitionKey, rowKey, creatorId, date, htmlText)
    {            
        this.StorageAccountName = storageAccountName;
        this.StorageContainerName = storageContainerName;            

        this.BlobName_Original = blobName_Original;
        this.BlobName_Large = blobName_Large;
        this.BlobName_Medium = blobName_Medium;
        this.BlobName_Small = blobName_Small;

        this.SetSubjectIdsList(subjectIds);
    }
}

So I call the InsertOperation like that (nothing special I think):

 ImagePostTableEntity newPost = new ImagePostTableEntity(streamId.ToString(), newPostId.ToString(), creatorId, date, htmlText, subjectIds, storageAccountName, storageContainerName, blobName_Original, blobName_Large, blobName_Medium, blobName_Small); //This construcotr calls inner method: SetSubjectIdsList(subjectIds);
            newPost.TotalComments = 0;
            newPost.VeryImportantData = "That very important string";
            TableOperation insertOperation = TableOperation.Insert(newPost);

After this operation an entity exists in table storage, but some Properties are not stored. To be specific only "SubjectIds" and "VeryImportantData" is not stored. They are not null and they have some value (double checked ;))

Answer 1

Weather you are facing issue with saving or after saving have missing attributes. First thing to notice is the TableOperation.Insert method itself. by checking its parameters, it will be obvious that it accepts a parameter of Microsoft.Azure.Cosmos.Table.ITableEntity type. Which means if you pass any object, it will be automatically casted to that interface, of course, as long as this objects comes from a class implements that interface.

namespace Microsoft.WindowsAzure.Storage.Table
{
    public interface ITableEntity
    {
        string PartitionKey { get; set; }
        string RowKey { get; set; }
        DateTimeOffset Timestamp { get; set; }
        string ETag { get; set; }

        void ReadEntity(IDictionary<string, EntityProperty> properties, OperationContext operationContext);
        IDictionary<string, EntityProperty> WriteEntity(OperationContext operationContext);
    }
}

by checking this interface members, I'm sure these two properties (PartitionKey, RowKey) are required and should be initialized before going to database

And to inform the runtime about the extra custom members in your object, this interface has a WriteEntity function. this function is used internally to return a collection with all the properties in the object.

Luckily or due to the best practices, TableEntity class decorated its implementation to the WriteEntity with virtual so any derived class can benefit from overriding it and add the needed properties

namespace Microsoft.WindowsAzure.Storage.Table
{
    public class TableEntity : ITableEntity
    {
        public TableEntity();
        public TableEntity(string partitionKey, string rowKey);

        public string PartitionKey { get; set; }
        public string RowKey { get; set; }
        public DateTimeOffset Timestamp { get; set; }
        public string ETag { get; set; }

        public static TResult ConvertBack<TResult>(IDictionary<string, EntityProperty> properties, OperationContext operationContext);
        public static TResult ConvertBack<TResult>(IDictionary<string, EntityProperty> properties, EntityPropertyConverterOptions entityPropertyConverterOptions, OperationContext operationContext);
        public static IDictionary<string, EntityProperty> Flatten(object entity, OperationContext operationContext);
        public static IDictionary<string, EntityProperty> Flatten(object entity, EntityPropertyConverterOptions entityPropertyConverterOptions, OperationContext operationContext);
        public static void ReadUserObject(object entity, IDictionary<string, EntityProperty> properties, OperationContext operationContext);
        public static IDictionary<string, EntityProperty> WriteUserObject(object entity, OperationContext operationContext);
        public virtual void ReadEntity(IDictionary<string, EntityProperty> properties, OperationContext operationContext);
        public virtual IDictionary<string, EntityProperty> WriteEntity(OperationContext operationContext);
    }
}

I made a sample for you. for sure the hardcoded values will be dynamic and will be calculated according to some logic in your app

public class CustomClass : TableEntity
{
    public CustomClass()
    {
        RowKey = Guid.NewGuid().ToString();
        PartitionKey = DateTime.Today.ToString("yyyy-MM");
        Timestamp = DateTime.Now;
        ETag = "*";
    }

    public CustomClass(string id, string name, bool? vip) : this()
    {
        Id = id;
        Name = name;
        VIP = vip;
    }

    public string Id { get; set; }

    public string Name { get; set; }

    public bool? VIP { get; set; }

    public override IDictionary<string, EntityProperty> WriteEntity(OperationContext operationContext)
    {
        return new Dictionary<string, EntityProperty>() {
            {nameof(Id), new EntityProperty(Id) },
            {nameof(Name), new EntityProperty(Name) },
            {nameof(VIP), new EntityProperty(VIP) },
            {nameof(RowKey), new EntityProperty(RowKey) },
            {nameof(Timestamp), new EntityProperty(Timestamp) },
            {nameof(ETag), new EntityProperty(ETag) },
            {nameof(PartitionKey), new EntityProperty(PartitionKey) }
        };
    }
}

Answer 2

In addition to what Emily mentioned about not having the execute method, I'm not seeing the code in which you set a value for subjectIds. Another good point Emily brought up, how are you verifying that the entity does not have those properties?

Try using fiddler to make sure that something is actually being sent.

I'm pretty sure you've seen this guidance before but check it out again to make sure you're not missing anything.

Remember, when doing an Insert operation, the entity will not be inserted if an entity with the same PK and RK already exists within that table. An InsertOrReplace operation will allow you to do this.