我目前有一个用&分隔的文本字符串，我需要使用PySpark解析以将键值对提取到数组/字典中。我可以使用字符串中的大多数标签来做到这一点，但问题是有些键有一个索引，这可能会因记录而异，但键将始终相同（如果这有意义的话。我想做的是，循环遍历字符串
示例输入：
"{pr1nm=Apples&pr1id=1111111&pr1pr=200.00&pr1qt=1&pr2nm=Pears&pr2id=1111112&pr2pr=300.00&pr2qt=2}"
期望输出：
[{
"ProductName":"Apples",
"ProductId": "1111111",
"ProductPrice":"200.00",
"ProductQuantity":"1",
"ProductName":"Pears",
"ProductId":"1111112",
"ProductPrice":"300.00"
"ProductQuantity":"2"
}]
此外，如果我在同一字符串中有其他标签，它们与产品无关，例如：
"{dl=https://stackoverflow.com/posts/XXXXX&t=pageview&pr1nm=Apples&pr1id=1111111&pr1pr=200.00&pr1qt=1&pr2nm=Pears&pr2id=1111112&pr2pr=300.00&pr2qt=2}"
嵌套数组中的产品的输出应该是这样的：
{"DocumentLocation":"https://stackoverflow.com/posts/XXXXX",
"HitType":"pageview",
"Products": [{
"ProductName":"Apples",
"ProductId": "1111111",
"ProductPrice":"200.00",
"ProductQuantity":"1",
"ProductName":"Pears",
"ProductId":"1111112",
"ProductPrice":"300.00"
"ProductQuantity":"2"
}]
}
您可以使用str_to_map将字符串转换为映射列，如下所示：
df = df.withColumn("input", expr("ltrim('{', rtrim('}', input))"))\
.withColumn("input", expr("str_to_map(input, '&', '=')"))
df.show(truncate=False)
+-------------------------------------------------------------------------------------------------------------------------------+
|input |
+-------------------------------------------------------------------------------------------------------------------------------+
|[pr1nm -> Apples, pr1id -> 1111111, pr1pr -> 200.00, pr1qt -> 1, pr2nm -> Pears, pr2id -> 1111112, pr2pr -> 300.00, pr2qt -> 2]|
+-------------------------------------------------------------------------------------------------------------------------------+
然后，如果你想要一个JSON字符串，使用to_json函数：
df.withColumn("input", to_json(col("input"))) \
.show(truncate=False)
+--------------------------------------------------------------------------------------------------------------------------------+
|input |
+--------------------------------------------------------------------------------------------------------------------------------+
|{"pr1nm":"Apples","pr1id":"1111111","pr1pr":"200.00","pr1qt":"1","pr2nm":"Pears","pr2id":"1111112","pr2pr":"300.00","pr2qt":"2"}|
+--------------------------------------------------------------------------------------------------------------------------------+
