通过Polybase加载Azure数据仓库时，我正在读取Azure Blob上的Parquet文件。

首先，我在SQL中创建了一个外部表来指向Parquet文件，然后使用CTAS加载。无论我在SQL中使用什么数据类型，它都会给我这种类型转换错误。我尝试过DECIMAL、NUMERIC、FLOAT。但是加载VARCHAR工作正常。

我怀疑这与Parquet文件是如何创建的有关，它来自PythonPandas数据帧，使用df.to_parquet和使用py箭头。深入源代码并进行实验，我看到箭头中的数据类型数据（Parquet之前的步骤）是Double。也许这就是为什么？

此外，我在创建文件时尝试了Gzip和Snappy作为压缩类型，在创建SQL外部表时，没有骰子。

快疯了。有什么想法吗？

conda create -n testenv python=3.6

conda install -n testenv -c conda-forge pyarrow

conda list -n testenv

# Name                    Version                   Build  Channel
arrow-cpp                 0.13.0           py36hee3af98_1    conda-forge
boost-cpp                 1.68.0            h6a4c333_1000    conda-forge
brotli                    1.0.7             he025d50_1000    conda-forge
ca-certificates           2019.3.9             hecc5488_0    conda-forge
certifi                   2019.3.9                 py36_0    conda-forge
gflags                    2.2.2             he025d50_1001    conda-forge
glog                      0.3.5                h6538335_1
intel-openmp              2019.3                      203
libblas                   3.8.0                     5_mkl    conda-forge
libcblas                  3.8.0                     5_mkl    conda-forge
liblapack                 3.8.0                     5_mkl    conda-forge
libprotobuf               3.7.1                h1a1b453_0    conda-forge
lz4-c                     1.8.1.2              h2fa13f4_0
mkl                       2019.3                      203
numpy                     1.16.2           py36h8078771_1    conda-forge
openssl                   1.1.1b               hfa6e2cd_2    conda-forge
pandas                    0.24.2           py36h6538335_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
pip                       19.0.3                   py36_0
pyarrow                   0.13.0           py36h8c67754_0    conda-forge
python                    3.6.8                h9f7ef89_7
python-dateutil           2.8.0                      py_0    conda-forge
pytz                      2019.1                     py_0    conda-forge
re2                       2019.04.01       vc14h6538335_0  [vc14]  conda-forge
setuptools                41.0.0                   py36_0
six                       1.12.0                py36_1000    conda-forge
snappy                    1.1.7             h6538335_1002    conda-forge
sqlite                    3.27.2               he774522_0
thrift-cpp                0.12.0            h59828bf_1002    conda-forge
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.33.1                   py36_0
wincertstore              0.2              py36h7fe50ca_0
zlib                      1.2.11            h2fa13f4_1004    conda-forge
zstd                      1.3.3                    vc14_1    conda-forge

>>> import pandas as pd
>>> df = pd.DataFrame({'ticker':['AAPL','AAPL','AAPL'],'price':[101,102,103]})
>>> df
  ticker  price
0   AAPL    101
1   AAPL    102
2   AAPL    103
>>> df.to_parquet('C:/aapl_test.parquet',engine='pyarrow',compression='snappy',index=False)

1. 将Parquet文件上传到Azure Blob
2. 使用Azure数据仓库Gen2，大小：DW400c
3. 根据文档和教程，创建了DATABASE SCOPED CREDENTIAL、外部数据源和外部文件格式

CREATE EXTERNAL FILE FORMAT [ParquetFileSnappy] WITH (
    FORMAT_TYPE = PARQUET, 
    DATA_COMPRESSION = N'org.apache.hadoop.io.compress.SnappyCodec'
)
GO

CREATE EXTERNAL DATA SOURCE [AzureBlobStorage] WITH (
    TYPE = HADOOP, 
    LOCATION = N'wasbs://[redacted: containerName]@[redacted: storageAccountName].blob.core.windows.net', 
    CREDENTIAL = [AzureQuantBlobStorageCredential] -- created earlier
)
GO

CREATE EXTERNAL TABLE ext.technicals(
    [ticker] VARCHAR(5) NOT NULL ,
    [close_px] DECIMAL(8,2) NULL
) WITH (
    LOCATION='/aapl_test.parquet', 
    DATA_SOURCE=AzureBlobStorage, 
    FILE_FORMAT=ParquetFileSnappy 
);

CREATE TABLE [dbo].TechnicalFeatures
WITH
( 
    DISTRIBUTION = ROUND_ROBIN,
    CLUSTERED COLUMNSTORE INDEX
)
AS SELECT * FROM [ext].technicals
OPTION (LABEL = 'CTAS : Load [dbo].[TechnicalFeatures]')
;

Msg 106000, Level 16, State 1, Line 20
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: ClassCastException: class java.lang.Long cannot be cast to class parquet.io.api.Binary (java.lang.Long is in module java.base of loader 'bootstrap'; parquet.io.api.Binary is in unnamed module of loader 'app')

编辑：还尝试使用fastparque而不是py箭头，同样的错误。