欢迎访问作物学报，今天是
作物学报

作物学报 ›› 2024, Vol. 50 ›› Issue (12): 3083-3095.doi: 10.3724/SP.J.1006.2024.42016

• 耕作栽培·生理生化 • 上一篇    下一篇

基于改进Pix2Pix-HD网络的多品种水稻生长可视化预测方法

段凌凤1(), 王新轶1, 王治昊1, 耿泽栋2, 卢运瑞2, 杨万能1,2,*()   

  1. 1华中农业大学工学院, 湖北武汉 430070
    2华中农业大学作物遗传改良国家重点实验室, 湖北武汉 430070
  • 收稿日期:2024-03-10 接受日期:2024-08-15 出版日期:2024-12-12 网络出版日期:2024-09-02
  • 通讯作者: *杨万能, E-mail: ywn@mail.hzau.edu.cn
  • 作者简介:E-mail: duanlingfeng@mail.hzau.edu.cn
  • 基金资助:
    国家自然科学基金项目(32170411);湖北洪山实验室开放课题(2021hskf005)

Growth visualized prediction method of multi-variety rice based on improved Pix2Pix-HD network

DUAN Ling-Feng1(), WANG Xin-Yi1, WANG Zhi-Hao1, GENG Ze-Dong2, LU Yun-Rui2, YANG Wan-Neng1,2,*()   

  1. 1College of Engineering, Huazhong Agricultural University, Wuhan 430070, Hubei, China
    2National Key Laboratory of Crop Genetic Improvement, Huazhong Agricultural University, Wuhan 430070, Hubei, China
  • Received:2024-03-10 Accepted:2024-08-15 Published:2024-12-12 Published online:2024-09-02
  • Contact: *E-mail: ywn@mail.hzau.edu.cn
  • Supported by:
    National Natural Science Foundation of China(32170411);Hubei Hongshan Laboratory Open Project(2021hskf005)

RichHTML

25

PDF

271

摘要：

植物生长建模与预测能模拟植物的生长过程, 有助于生理学家和植物学家分析植物未来的生长模式, 缩短试验周期、降低试验成本, 受时间和条件限制的植物试验与研究指导。生长可视化预测能提供未来生长时间点的植物图像, 能更逼真、直观地描述植物的生长过程。水稻作为重要的粮食作物, 实现水稻的生长可视化预测, 对水稻生长发育分析具有十分重要的意义。针对传统作物生长预测方法存在的视觉真实度和可视化效果较差等问题, 本文提出了一种基于改进Pix2Pix-HD模型的多品种水稻生长可视化预测方法, 利用数据驱动的方式, 实现了对水稻抽穗期到灌浆期的高分辨率生长可视化预测, 通过水稻抽穗期的图像预测灌浆期水稻生长图像。方法评估中,本文从视觉相似性、表型准确性和不同尺度评估模型预测性能, 通过消融实验评估改进方法的有效性, 并与现有研究进行比较。结果表明,测试集预测的灌浆期水稻图像与真实灌浆期水稻图像之间的FID、PSNR和SSIM值分别达到24.75、13.58和0.78, 预测表型和真实表型相关系数的平均值为0.762, 在不同尺度上都能保持较好的准确性。本文提出的基于数据驱动的水稻生长预测方法能够实现高分辨率和高视觉真实性的水稻生长可视化预测, 为水稻生长预测提供了新思路。

关键词: 水稻, 生长可视化预测, 生成对抗网络, Pix2Pix-HD, 数字植物

Abstract:

With the advancement of information technology and agricultural digitization, the visualized prediction of digital plant growth holds significant development potential. As an important food crop, achieving visualized growth prediction for rice is of great significance for analyzing its growth and development. However, due to the large size and complex morphology of rice, attaining high-precision and high-resolution visualized growth prediction has been challenging. This study proposes a growth prediction method for rice based on an improved Pix2Pix-HD network, achieving high-precision visualized growth prediction from the heading stage to the filling stage of rice. Comparative experiments were designed to verify the effectiveness of the model improvement scheme. The results showed that the FID, PSNR, and SSIM values between the predicted and real filling stage rice images in the test set were 24.75, 13.58, and 0.78, respectively. The average correlation coefficient between the predicted and actual phenotypes was 0.762, demonstrating good accuracy at different scales. The proposed data-driven rice growth prediction method can achieve high-resolution and high visual realism in rice growth visualization predictions, providing new solutions for rice growth forecasting.

Key words: rice, growth visualized prediction, generative adversarial network, Pix2Pix-HD, digital plants

图1

总体技术流程 RGVP: 本研究提出的水稻生长可视化预测模型; Pix2Pix-HD: 基于深度学习的用于图像转换的生成对抗网络模型。"

图2

Pix2Pix-HD网络结构 G1: 全局生成器; G2a: 局部增强器的第1部分; G2b: 局部增强器的第2部分; D1: 1024×1024分辨率尺度判别器; D2: 512×512分辨率尺度判别器; D3: 256×256分辨率尺度判别器。"

图3

改进Pix2Pix-HD中全局生成器和判别器网络结构 G1: 全局生成器; D1: 1024×1024分辨率尺度判别器; RefP: 反射填充层; Conv: 卷积层; BN: 批归一化层; ReLU、LeakyReLU、Tanh、Sigmoid: 不同的激活函数; ConvT: 反卷积层; Dropout: 一种正则化方法。"

图4

预测结果示例 A1、A2和A3: 抽穗期水稻图像; B1、B2和B3: 预测灌浆期水稻图像; C1、C2和C3: 真实灌浆期水稻图像。"

表1

预测表型与真实表型的相关系数"

评价指标
Evaluation indicators		 形态特征相关系数
Correlation coefficient of
morphological traits		 纹理特征相关系数
Correlation coefficient of texture traits		 MCCM MCCT MCCA
TPA GPA YPA FD H W 直方图特征
Histogram		 灰度梯度共生矩阵特征
Gray-level co-occurrence matrix
M SE S MU3 U E C GM GDM GMSE GDMSE
CC 0.77 0.69 0.82 0.78 0.58 0.53 0.84 0.86 0.85 0.86 0.77 0.79 0.79 0.62 0.80 0.80 0.81 0.70 0.80 0.76
CCS 0.15 0.21 -0.05 0.17 0.01 0.27 0.06 0.03 0.06 -0.02 0.16 0.16 -0.08 -0.08 -0.05 -0.01 -0.02 0.13 0.02 0.06

表2

预测图像与真实图像不同尺度对比"

分辨率
Resolution		 视觉相似性Visualized similarity 表型准确性Phenotypic accuracy
Fréchet距离FID 峰值信噪比PSNR 结构相似度SSIM MCCM MCCT MCCA
1024×1024 24.75 13.58 0.78 0.70 0.80 0.76
512×512 21.51 14.47 0.73 0.69 0.78 0.75
256×256 40.69 15.77 0.68 0.67 0.70 0.69

表3

不同改进方案模型性能比较"

模型 Model 模型改进设计方案
Model improvement		 视觉相似性
Visualized similarity		 表型准确性
Phenotypic accuracy
训练阶段
随机失活
Training dropout		 测试阶段
随机失活
Testing dropout		 假标签损失
Fake label loss		 空间注意力
机制
Spatial attention		 通道注意力
机制
Channel
attention		 Fréchet距离
FID		 峰值信噪比 PSNR 结构相似度 SSIM MCCM MCCT MCCA
模型1
Model 1		 √ √ × × × 30.44 13.27 0.77 0.64 0.72 0.69
模型2
Model 2		 × × × × × 26.10 13.76 0.78 0.62 0.74 0.70
模型3
Model 3		 √ × × × × 37.60 13.46 0.78 0.66 0.78 0.74
模型4
Model 4		 √ × √ × × 29.65 13.58 0.78 0.65 0.78 0.74
模型5
Model 5		 √ × √ √ × 24.75 13.58 0.78 0.70 0.80 0.76
模型6
Model 6		 √ × √ √ √ 32.80 13.72 0.78 0.68 0.81 0.76

图5

Pix2Pix模型和RGVP模型的预测结果对比 A1、A2和A3: 抽穗期水稻图像; B1、B2和B3: Pix2Pix模型预测灌浆期水稻图像; C1、C2和C3: RGVP模型预测灌浆期水稻图像; D1、D2和D3: 真实灌浆期水稻图像; Pix2Pix: 基于深度学习的用于图像转换的生成对抗网络模型; RGVP: 本研究提出的水稻生长可视化预测模型。"

表4

Pix2Pix模型与RGVP模型在2种尺度的对比"

模型及处理
Model and process		 分辨率
Resolution		 视觉相似性Visualized similarity 表型准确性Phenotypic accuracy
Fréchet距离
FID		 峰值信噪比
PSNR		 结构相似度
SSIM		 MCCM MCCT MCCA
Pix2Pix模型
Pix2Pix model		 256×256 45.60 14.53 0.65 0.50 0.63 0.58
RGVP + 1/4下采样
RGVP + 1/4 Down-sampling		 256×256 40.69 15.77 0.68 0.67 0.70 0.69
Pix2Pix + 4倍超分辨
Pix2Pix + 4× super-resolution		 1024×1024 68.91 13.31 0.76 0.49 0.64 0.59
RGVP模型
RGVP model		 1024×1024 24.75 13.58 0.78 0.70 0.80 0.76
[1] 梁伟军. 农业与相关产业融合发展研究. 华中农业大学博士学位论文, 湖北武汉, 2010.
Liang W J. Research on the Integration and Development of Agriculture and Related Industries. PhD Dissertation of Huazhong Agricultural University, Wuhan, Hubei, China, 2010 (in Chinese with English abstract).
[2] 赵春江, 陆声链, 郭新宇, 肖伯祥, 温维亮. 数字植物及其技术体系探讨. 中国农业科学, 2010, 43: 2023-2030.
Zhao C J, Lu S L, Guo X Y, Xiao B X, Wen W L. Exploration of digital plants and their technical system. Sci Agric Sin, 2010, 43: 2023-2030 (in Chinese with English abstract).
[3] 张洪程, 王夫玉. 中国水稻群体研究进展. 中国水稻科学, 2001, 15: 51-56.
Zhang H C, Wang F Y. Research progress on rice population in China. Chin J Rice Sci, 2001, 15: 51-56 (in Chinese with English abstract).
[4] 程式华, 廖西元, 闵绍楷. 中国超级稻研究: 背景、目标和有关问题的思考. 中国稻米, 1998, (1): 3-5.
Cheng S H, Liao X Y, Min S K. Research on Chinese super rice: background, objectives, and reflections on related issues. China Rice, 1998, (1): 3-5 (in Chinese).
[5] 邹应斌, 周上游, 唐起源. 中国超级杂交水稻超高产栽培研究的现状与展望. 中国农业科技导报, 2003, 5(1): 31-35.
Zou Y B, Zhou S Y, Tang Q Y. The current status and prospects of research on super high yield cultivation of Chinese super hybrid rice. J Agric Sci Technol, 2003, 5(1): 31-35 (in Chinese with English abstract).
[6] Jung K H, An G, Ronald P C. Towards a better bowl of rice: assigning function to tens of thousands of rice genes. Nat Rev Genet, 2008, 9: 91-101.
[7] Alhnaity B, Kollias S, Leontidis G, Jiang S Y, Schamp B, Pearson S. An autoencoder wavelet based deep neural network with attention mechanism for multi-step prediction of plant growth. Inform Sci, 2021, 560: 35-50.
[8] Alhnaity B, Pearson S, Leontidis G, Kollias S. Using deep learning to predict plant growth and yield in greenhouse environments. Int Soc Hortic Sci, 2019, 1296: 425-432.
[9] 张慧春, 王国苏, 边黎明, 郑加强, 周宏平. 基于光学相机的植物表型测量系统与时序生长模型研究. 农业机械学报, 2019, 50(10): 197-207.
Zhang H C, Wang G S, Bian L M, Zheng J Q, Zhou H P. Research on plant phenotype measurement system and temporal growth model based on optical cameras. Trans CSAE, 2019, 50(10): 197-207 (in Chinese with English abstract).
[10] 朱新广, 常天根, 宋青峰, 常硕其, 王重荣, 张国庆, 郭亚, 周少川. 数字植物: 科学内涵、瓶颈及发展策略. 合成生物学, 2020, 1: 285-297.
doi: 10.12211/2096-8280.2020-018
Zhu X G, Chang T G, Song Q F, Chang S Q, Wang C R, Zhang G Q, Guo Y, Zhou S C. Plants: scientific connotation, bottlenecks, and development strategies. Syn Biol J, 2020, 1: 285-297 (in Chinese with English abstract).
[11] Lindenmayer A. Mathematical models for cellular interactions in development. J Theor Biol, 1968, 18: 300-315.
pmid: 5659072
[12] Leitner D, Klepsch S, Knie A, Schnepf A. The algorithmic beauty of plant roots: an L-System model for dynamic root growth simulation. Math Comput Model Dyn Sys, 2010, 16: 575-587.
[13] Espana M, Baret F, Aries F, Chelle M, Andrieu B, Prevot L. Modeling maize canopy 3D architecture: application to reflectance simulation. Ecol Model, 1999, 122: 25-43.
[14] Jallas E, Sequeira R, Martin P, Turner S, Papajorgji P. Mechanistic cirtual modeling: coupling a plant simulation model with a three-dimensional plant architecture component. Environ Model Assess, 2009, 14: 29-45.
[15] Qian B, Huang W J, Xie D H, Ye H C, Guo A, Pan Y H, Jin Y, Xie Q Y, Jiao Q J, Zhang B Y, Ruan C, Xu T J, Zhang Y, Nie T G. Coupled maize model: a 4D maize growth model based on growing degree days. Comput Electron Agric, 2023, 212: 108124.
[16] Minorsky P V. Achieving the in silico plant. Systems biology and the future of plant biological research. Plant Physiol, 2003, 132: 404-409.
pmid: 12822566
[17] Prusinkiewicz P. Modeling plant growth and development. Curr Opin Plant Biol, 2004, 7: 79-83.
pmid: 14732445
[18] 肖亚, 李玉强. 农业人工智能综述. 数字技术与应用, 2020, 38(9): 204-205.
Xiao Y, Li Y Q. Overview of agricultural artificial intelligence. Digit Technol Appl, 2020, 38(9): 204-205 (in Chinese with English abstract).
[19] Fahlgren N, Gehan M A, Baxter I. Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. Curr Opin Plant Biol, 2015, 24: 93-99.
doi: 10.1016/j.pbi.2015.02.006 pmid: 25733069
[20] Kim T H, Lee S H, Oh M M, Kim J O. Plant growth prediction based on hierarchical auto-encoder. In: International Conference on Electronics, Information, and Communication. Jeju: IEEE, 2022. pp 1-3.
[21] Kim T H, Lee S H, Kim J O. A novel shape based plant growth prediction algorithm using deep learning and spatial transformation. IEEE Access, 2022, 10: 37731-37742.
[22] Sakurai S, Uchiyama H, Shimada A, Taniguchi R. Plant growth prediction using convolutional LSTM. VISIGRAPP, 2019, 14: 105-113.
[23] 王春颖, 泮玮婷, 李祥, 刘平. 基于STLSTM的植物生长发育预测模型. 农业机械学报, 2022, 53(6): 250-258.
Wang C Y, Pan W T, Li X, Liu P. A plant growth and development prediction model based on STLSTM. Trans CSAM, 2022, 53(6): 250-258 (in Chinese with English abstract).
[24] Wang C, Pan W, Song X, Yu H, Zhu J, Liu P, Li X. Predicting plant growth and development using time-series images. Agronomy, 2022, 12: 2213.
[25] Yasrab R, Zhang J, Smyth P, Pound M P. Predicting plant growth from time-series data using deep learning. Remote Sens, 2021, 13: 331.
[26] Drees L, Junker-Frohn L V, Kierdorf J, Roscher R. Temporal prediction and evaluation of Brassica growth in the field using conditional generative adversarial networks. Comput Electron Agric, 2021, 190: 106415.
[27] Meng Y, Xu M, Yoon S, Jeong Y, Park D S. Flexible and high-quality plant growth prediction with limited data. Front Plant Sci, 2022, 13: 989304.
[28] Yang W, Guo Z, Huang C, Duan L, Chen G, Jiang N, Fang W, Feng H, Xie W, Lian X. Combining high-throughput phenotyping and genome-wide association studies to reveal natural genetic variation in rice. Nat Commun, 2014, 5: 5087.
doi: 10.1038/ncomms6087 pmid: 25295980
[29] Wang T C, Liu M Y, Zhu J Y, Tao A, Kautz J, Catanzaro B. High-resolution image synthesis and semantic manipulation with conditional GANs. Comput Vis Pattern Recognit, 2017. pp 1-14.
[30] Woo S, Park J, Lee J Y, Kweon I S. CBAM:convolutional block attention module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Proceedings of the European Conference on Computer Vision. Munich: Springer Cham, 2018. pp 3-19.
[31] Dowson D C, Landau B V. The fréchet distance between multivariate normal distributions. J Multivariate Anal, 1982, 12: 450-455.
[32] Korhonen J, You J. Peak signal-to-noise ratio revisited: is simple beautiful?. In: Fourth International Workshop on Quality of Multimedia Experience. Melbourne: IEEE, 2012. pp 37-38.
[33] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proc, 2004, 13: 600-612.
[34] Wang X, Xie L, Dong C, Shan Y. Real-esrgan:training real-world blind super-resolution with pure synthetic data. In:Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. pp 1905-1914.
[1] 胡赵, 钱润, 谢丰璞, 应素平. 水稻SPX基因家族鉴定及响应磷处理的表达分析[J]. 作物学报, 2026, 52(6): 1902-1912.
[2] 邹仪妹, 徐敏, 汪海洋, 姚辉, 王加峰, 刘浩, 任代胜. 两系不育系水稻幼苗根系响应盐胁迫的转录因子调控网络鉴定[J]. 作物学报, 2026, 52(6): 1728-1742.
[3] 闫安, 蒋昆炜, 王蓉圆, 田林, 张璐, 王韵, 徐建龙. 水稻剑叶小维管束数基因SVN7的鉴定与克隆[J]. 作物学报, 2026, 52(5): 1364-1372.
[4] 陈伟, 卫万娟, 赵其兵, 常东伟, 余凌波, 翟鹏飞, 冯志明, 陈宗祥, 任仰涛, 杨鹏, 刘海浪, 李珍富, 杨永乐, 金彦刚, 左示敏. 利用CRISPR/Cas9编辑Hd6基因创制优质早熟水稻新种质[J]. 作物学报, 2026, 52(4): 1046-1056.
[5] 石少阶, 刘凯, 陈姿夷, 王卉颖, 李三和, 周雷, 游艾青. 水稻矮化多分蘖基因DMT1的克隆与功能分析[J]. 作物学报, 2026, 52(4): 1022-1034.
[6] 覃奕琰, 付瑶, 苏畅, 李娜, 徐静茹, 程笑然, 张琪, 赵明辉. OsST41调控水稻苗期耐盐性的功能分析[J]. 作物学报, 2026, 52(3): 802-812.
[7] 叶凡, 李帅, 李思宇, 陈云, 窦超银, 刘立军. 不同节水灌溉方式对东北稻区水稻产量和群体质量的影响[J]. 作物学报, 2026, 52(3): 895-907.
[8] 王婵, 吴莹莹, 李文奇, 李霞, 王芳权, 周彤, 杨杰. 基于HRM技术开发水稻抗条纹叶枯病基因STV11功能标记[J]. 作物学报, 2025, 51(9): 2547-2556.
[9] 陈惠莹, 何嘉欣, 朱斌, 黄士轩, 周星佑, 伍君权, 杨美艳. 水稻黄单胞菌噬菌体vB_XaS_HDB2的全基因组分析和生物学特性研究[J]. 作物学报, 2025, 51(8): 2087-2099.
[10] 杨海洋, 吴林宣, 李博纹, 石翰峰, 袁禧龙, 刘金朝, 蔡海荣, 陈诗怡, 郭涛, 王慧. 基于QTL定位发现的OsWRI3调控水稻种子的落粒性[J]. 作物学报, 2025, 51(7): 1712-1724.
[11] 雷松翰, 范骏扬, 车艳奕, 代永东, 郑雨萌, 田维江, 桑贤春, 王晓雯. 水稻内卷叶突变体acl3的鉴定及调控基因的功能分析[J]. 作物学报, 2025, 51(6): 1467-1479.
[12] 李福媛, 杨奕, 马继琼, 许明辉, 林良斌, 孙一丁. 水稻OsPUB4基因克隆、激素诱导表达分析与互作蛋白筛选[J]. 作物学报, 2025, 51(6): 1690-1700.
[13] 王梦宁, 谢可冉, 高逖, 王飞, 任孝俭, 熊栋梁, 黄见良, 彭少兵, 崔克辉. 水稻幼穗分化期至抽穗期高温对籽粒形态和充实的影响及其与粒重的关系[J]. 作物学报, 2025, 51(5): 1347-1362.
[14] 盛倩男, 方娅婷, 赵剑, 杜思垚, 胡行珍, 余秋华, 朱俊, 任涛, 鲁剑巍. 不同养分管理措施对稻田和旱地油菜产量的影响及其对冻害的响应[J]. 作物学报, 2025, 51(5): 1286-1298.
[15] 翁文安, 邢志鹏, 胡群, 魏海燕, 廖萍, 朱海滨, 瞿济伟, 李秀丽, 刘桂云, 高辉, 张洪程. 无人化旱直播水稻产量形成特征及其能量与经济效益研究[J]. 作物学报, 2025, 51(5): 1363-1377.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备15008789号-2