我曾经问过类似的问题,所以请原谅,如果你发现它更多的重复,虽然这里的代码是不同的,我正在努力修复。
下面是我正在读取JSON文件并复制Azure表格存储中的内容的代码。正在从blob存储读取Json文件。现在我正在从记忆中阅读并传递要复制的内容。但是考虑到大型Json文件可能会给我带来内存异常,我希望将其作为流读取,而不将其存储为内存。我该怎么做呢?
读取JSON的代码:
List<string> lines = new List<string>();
foreach (var files in recFiles)
{
Stream data = await DownloadBlob(containerName, fileName, connectionString);
StreamReader reader = new StreamReader(data, Encoding.UTF8);
string dataContents = reader.ReadToEnd();
lines.Add(dataContents);
}
await PopulateTable(lines);
DownloadBlob:
public async Task<Stream> DownloadBlob(string containerName, string fileName, string connectionString)
{
Microsoft.Azure.Storage.CloudStorageAccount storageAccount = Microsoft.Azure.Storage.CloudStorageAccount.Parse(connectionString);
CloudBlobClient serviceClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = serviceClient.GetContainerReference(containerName);
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
if (!blob.Exists())
{
throw new Exception("Blob Not found");
}
return await blob.OpenReadAsync();
}
读取和上传Json:
public async Task<List<DynamicTableEntity>> PopulateTable(IEnumerable<string> lines)
{
var validator = new JsonSchemaValidator();
var tableData = new List<JObject>();
// Validate all entries
foreach (var line in lines)
{
if (string.IsNullOrWhiteSpace(line))
continue;
var data = JsonConvert.DeserializeObject<JObject>(line);
...... // adding to table
}
}
像这样的办法可能会管用。我假设这是一个更多的项目阵列,我没有时间完全测试它。
List<string> lines = new List<string>();
foreach (var files in recFiles)
{
using (Stream data = await DownloadBlob(containerName, fileName, connectionString))
{
using (StreamReader reader = new StreamReader(data, Encoding.UTF8) )
{
while(reader.Read())
{
if (reader.TokenType == JsonToken.StartObject)
{
// load each object from stream
var jobject = JObject.Load(reader);
PopulateTable(jobject.ToString(Formatting.None));
}
}
}
}
}