I have a requirement to send xml files between 2 servers. These files could be up to 1mb.
I was considering using an Azure Storage Queue as an intermediary destination for these files but read that there is a 64kb limit on message size. I would imagine the requirement to send messages in excess of 64kb is not uncommon.
Is there a way round the limit? Compressing perhaps?
Queueing is not storage.
Save the payload in azure blob storage then send a message in queue with a link to the blob.
First, Azure Storage Queue isn't for storing files. Storage Queues are for sending messages. Yes, there's a somewhat small message size limitation for performance reasons.
Is there a way round the limit?
Azure Storage Queue isn't meant for sending large amounts of data through as messages. To send larger amounts of data you would need to store it somewhere like in an Azure Storage account with a pointer to the location of where the file is stored within the message sent to the Azure Storage Queue. This allows for the sender application to wrap everything up in a way that the receiver application will be able to get the message and be able to tell where the associated file is located.
Compressing perhaps?
Using compression to make the file smaller to fit within the Queue Message limitation isn't really reliable enough. Sometimes you may get files that are still too large when compressed and those messages will throw exception when attempting to send. The best way is to use the method described above.
