欢迎访问作物学报，今天是
作物学报

作物学报 ›› 2024, Vol. 50 ›› Issue (12): 3083-3095.doi: 10.3724/SP.J.1006.2024.42016

• 耕作栽培·生理生化 • 上一篇    下一篇

基于改进Pix2Pix-HD网络的多品种水稻生长可视化预测方法

段凌凤1(), 王新轶1, 王治昊1, 耿泽栋2, 卢运瑞2, 杨万能1,2,*()   

  1. 1华中农业大学工学院, 湖北武汉 430070
    2华中农业大学作物遗传改良国家重点实验室, 湖北武汉 430070
  • 收稿日期:2024-03-10 接受日期:2024-08-15 出版日期:2024-12-12 网络出版日期:2024-09-02
  • 通讯作者: *杨万能, E-mail: ywn@mail.hzau.edu.cn
  • 作者简介:E-mail: duanlingfeng@mail.hzau.edu.cn
  • 基金资助:
    国家自然科学基金项目(32170411);湖北洪山实验室开放课题(2021hskf005)

Growth visualized prediction method of multi-variety rice based on improved Pix2Pix-HD network

DUAN Ling-Feng1(), WANG Xin-Yi1, WANG Zhi-Hao1, GENG Ze-Dong2, LU Yun-Rui2, YANG Wan-Neng1,2,*()   

  1. 1College of Engineering, Huazhong Agricultural University, Wuhan 430070, Hubei, China
    2National Key Laboratory of Crop Genetic Improvement, Huazhong Agricultural University, Wuhan 430070, Hubei, China
  • Received:2024-03-10 Accepted:2024-08-15 Published:2024-12-12 Published online:2024-09-02
  • Contact: *E-mail: ywn@mail.hzau.edu.cn
  • Supported by:
    National Natural Science Foundation of China(32170411);Hubei Hongshan Laboratory Open Project(2021hskf005)

RichHTML

1

PDF

50

摘要：

植物生长建模与预测能模拟植物的生长过程, 有助于生理学家和植物学家分析植物未来的生长模式, 缩短试验周期、降低试验成本, 受时间和条件限制的植物试验与研究指导。生长可视化预测能提供未来生长时间点的植物图像, 能更逼真、直观地描述植物的生长过程。水稻作为重要的粮食作物, 实现水稻的生长可视化预测, 对水稻生长发育分析具有十分重要的意义。针对传统作物生长预测方法存在的视觉真实度和可视化效果较差等问题, 本文提出了一种基于改进Pix2Pix-HD模型的多品种水稻生长可视化预测方法, 利用数据驱动的方式, 实现了对水稻抽穗期到灌浆期的高分辨率生长可视化预测, 通过水稻抽穗期的图像预测灌浆期水稻生长图像。方法评估中,本文从视觉相似性、表型准确性和不同尺度评估模型预测性能, 通过消融实验评估改进方法的有效性, 并与现有研究进行比较。结果表明,测试集预测的灌浆期水稻图像与真实灌浆期水稻图像之间的FID、PSNR和SSIM值分别达到24.75、13.58和0.78, 预测表型和真实表型相关系数的平均值为0.762, 在不同尺度上都能保持较好的准确性。本文提出的基于数据驱动的水稻生长预测方法能够实现高分辨率和高视觉真实性的水稻生长可视化预测, 为水稻生长预测提供了新思路。

关键词: 水稻, 生长可视化预测, 生成对抗网络, Pix2Pix-HD, 数字植物

Abstract:

With the advancement of information technology and agricultural digitization, the visualized prediction of digital plant growth holds significant development potential. As an important food crop, achieving visualized growth prediction for rice is of great significance for analyzing its growth and development. However, due to the large size and complex morphology of rice, attaining high-precision and high-resolution visualized growth prediction has been challenging. This study proposes a growth prediction method for rice based on an improved Pix2Pix-HD network, achieving high-precision visualized growth prediction from the heading stage to the filling stage of rice. Comparative experiments were designed to verify the effectiveness of the model improvement scheme. The results showed that the FID, PSNR, and SSIM values between the predicted and real filling stage rice images in the test set were 24.75, 13.58, and 0.78, respectively. The average correlation coefficient between the predicted and actual phenotypes was 0.762, demonstrating good accuracy at different scales. The proposed data-driven rice growth prediction method can achieve high-resolution and high visual realism in rice growth visualization predictions, providing new solutions for rice growth forecasting.

Key words: rice, growth visualized prediction, generative adversarial network, Pix2Pix-HD, digital plants

图1

总体技术流程 RGVP: 本研究提出的水稻生长可视化预测模型; Pix2Pix-HD: 基于深度学习的用于图像转换的生成对抗网络模型。"

图2

Pix2Pix-HD网络结构 G1: 全局生成器; G2a: 局部增强器的第1部分; G2b: 局部增强器的第2部分; D1: 1024×1024分辨率尺度判别器; D2: 512×512分辨率尺度判别器; D3: 256×256分辨率尺度判别器。"

图3

改进Pix2Pix-HD中全局生成器和判别器网络结构 G1: 全局生成器; D1: 1024×1024分辨率尺度判别器; RefP: 反射填充层; Conv: 卷积层; BN: 批归一化层; ReLU、LeakyReLU、Tanh、Sigmoid: 不同的激活函数; ConvT: 反卷积层; Dropout: 一种正则化方法。"

图4

预测结果示例 A1、A2和A3: 抽穗期水稻图像; B1、B2和B3: 预测灌浆期水稻图像; C1、C2和C3: 真实灌浆期水稻图像。"

表1

预测表型与真实表型的相关系数"

评价指标
Evaluation indicators		 形态特征相关系数
Correlation coefficient of
morphological traits		 纹理特征相关系数
Correlation coefficient of texture traits		 MCCM MCCT MCCA
TPA GPA YPA FD H W 直方图特征
Histogram		 灰度梯度共生矩阵特征
Gray-level co-occurrence matrix
M SE S MU3 U E C GM GDM GMSE GDMSE
CC 0.77 0.69 0.82 0.78 0.58 0.53 0.84 0.86 0.85 0.86 0.77 0.79 0.79 0.62 0.80 0.80 0.81 0.70 0.80 0.76
CCS 0.15 0.21 -0.05 0.17 0.01 0.27 0.06 0.03 0.06 -0.02 0.16 0.16 -0.08 -0.08 -0.05 -0.01 -0.02 0.13 0.02 0.06

表2

预测图像与真实图像不同尺度对比"

分辨率
Resolution		 视觉相似性Visualized similarity 表型准确性Phenotypic accuracy
Fréchet距离FID 峰值信噪比PSNR 结构相似度SSIM MCCM MCCT MCCA
1024×1024 24.75 13.58 0.78 0.70 0.80 0.76
512×512 21.51 14.47 0.73 0.69 0.78 0.75
256×256 40.69 15.77 0.68 0.67 0.70 0.69

表3

不同改进方案模型性能比较"

模型 Model 模型改进设计方案
Model improvement		 视觉相似性
Visualized similarity		 表型准确性
Phenotypic accuracy
训练阶段
随机失活
Training dropout		 测试阶段
随机失活
Testing dropout		 假标签损失
Fake label loss		 空间注意力
机制
Spatial attention		 通道注意力
机制
Channel
attention		 Fréchet距离
FID		 峰值信噪比 PSNR 结构相似度 SSIM MCCM MCCT MCCA
模型1
Model 1		 √ √ × × × 30.44 13.27 0.77 0.64 0.72 0.69
模型2
Model 2		 × × × × × 26.10 13.76 0.78 0.62 0.74 0.70
模型3
Model 3		 √ × × × × 37.60 13.46 0.78 0.66 0.78 0.74
模型4
Model 4		 √ × √ × × 29.65 13.58 0.78 0.65 0.78 0.74
模型5
Model 5		 √ × √ √ × 24.75 13.58 0.78 0.70 0.80 0.76
模型6
Model 6		 √ × √ √ √ 32.80 13.72 0.78 0.68 0.81 0.76

图5

Pix2Pix模型和RGVP模型的预测结果对比 A1、A2和A3: 抽穗期水稻图像; B1、B2和B3: Pix2Pix模型预测灌浆期水稻图像; C1、C2和C3: RGVP模型预测灌浆期水稻图像; D1、D2和D3: 真实灌浆期水稻图像; Pix2Pix: 基于深度学习的用于图像转换的生成对抗网络模型; RGVP: 本研究提出的水稻生长可视化预测模型。"

表4

Pix2Pix模型与RGVP模型在2种尺度的对比"

模型及处理
Model and process		 分辨率
Resolution		 视觉相似性Visualized similarity 表型准确性Phenotypic accuracy
Fréchet距离
FID		 峰值信噪比
PSNR		 结构相似度
SSIM		 MCCM MCCT MCCA
Pix2Pix模型
Pix2Pix model		 256×256 45.60 14.53 0.65 0.50 0.63 0.58
RGVP + 1/4下采样
RGVP + 1/4 Down-sampling		 256×256 40.69 15.77 0.68 0.67 0.70 0.69
Pix2Pix + 4倍超分辨
Pix2Pix + 4× super-resolution		 1024×1024 68.91 13.31 0.76 0.49 0.64 0.59
RGVP模型
RGVP model		 1024×1024 24.75 13.58 0.78 0.70 0.80 0.76
[1] 梁伟军. 农业与相关产业融合发展研究. 华中农业大学博士学位论文, 湖北武汉, 2010.
Liang W J. Research on the Integration and Development of Agriculture and Related Industries. PhD Dissertation of Huazhong Agricultural University, Wuhan, Hubei, China, 2010 (in Chinese with English abstract).
[2] 赵春江, 陆声链, 郭新宇, 肖伯祥, 温维亮. 数字植物及其技术体系探讨. 中国农业科学, 2010, 43: 2023-2030.
Zhao C J, Lu S L, Guo X Y, Xiao B X, Wen W L. Exploration of digital plants and their technical system. Sci Agric Sin, 2010, 43: 2023-2030 (in Chinese with English abstract).
[3] 张洪程, 王夫玉. 中国水稻群体研究进展. 中国水稻科学, 2001, 15: 51-56.
Zhang H C, Wang F Y. Research progress on rice population in China. Chin J Rice Sci, 2001, 15: 51-56 (in Chinese with English abstract).
[4] 程式华, 廖西元, 闵绍楷. 中国超级稻研究: 背景、目标和有关问题的思考. 中国稻米, 1998, (1): 3-5.
Cheng S H, Liao X Y, Min S K. Research on Chinese super rice: background, objectives, and reflections on related issues. China Rice, 1998, (1): 3-5 (in Chinese).
[5] 邹应斌, 周上游, 唐起源. 中国超级杂交水稻超高产栽培研究的现状与展望. 中国农业科技导报, 2003, 5(1): 31-35.
Zou Y B, Zhou S Y, Tang Q Y. The current status and prospects of research on super high yield cultivation of Chinese super hybrid rice. J Agric Sci Technol, 2003, 5(1): 31-35 (in Chinese with English abstract).
[6] Jung K H, An G, Ronald P C. Towards a better bowl of rice: assigning function to tens of thousands of rice genes. Nat Rev Genet, 2008, 9: 91-101.
[7] Alhnaity B, Kollias S, Leontidis G, Jiang S Y, Schamp B, Pearson S. An autoencoder wavelet based deep neural network with attention mechanism for multi-step prediction of plant growth. Inform Sci, 2021, 560: 35-50.
[8] Alhnaity B, Pearson S, Leontidis G, Kollias S. Using deep learning to predict plant growth and yield in greenhouse environments. Int Soc Hortic Sci, 2019, 1296: 425-432.
[9] 张慧春, 王国苏, 边黎明, 郑加强, 周宏平. 基于光学相机的植物表型测量系统与时序生长模型研究. 农业机械学报, 2019, 50(10): 197-207.
Zhang H C, Wang G S, Bian L M, Zheng J Q, Zhou H P. Research on plant phenotype measurement system and temporal growth model based on optical cameras. Trans CSAE, 2019, 50(10): 197-207 (in Chinese with English abstract).
[10] 朱新广, 常天根, 宋青峰, 常硕其, 王重荣, 张国庆, 郭亚, 周少川. 数字植物: 科学内涵、瓶颈及发展策略. 合成生物学, 2020, 1: 285-297.
doi: 10.12211/2096-8280.2020-018
Zhu X G, Chang T G, Song Q F, Chang S Q, Wang C R, Zhang G Q, Guo Y, Zhou S C. Plants: scientific connotation, bottlenecks, and development strategies. Syn Biol J, 2020, 1: 285-297 (in Chinese with English abstract).
[11] Lindenmayer A. Mathematical models for cellular interactions in development. J Theor Biol, 1968, 18: 300-315.
pmid: 5659072
[12] Leitner D, Klepsch S, Knie A, Schnepf A. The algorithmic beauty of plant roots: an L-System model for dynamic root growth simulation. Math Comput Model Dyn Sys, 2010, 16: 575-587.
[13] Espana M, Baret F, Aries F, Chelle M, Andrieu B, Prevot L. Modeling maize canopy 3D architecture: application to reflectance simulation. Ecol Model, 1999, 122: 25-43.
[14] Jallas E, Sequeira R, Martin P, Turner S, Papajorgji P. Mechanistic cirtual modeling: coupling a plant simulation model with a three-dimensional plant architecture component. Environ Model Assess, 2009, 14: 29-45.
[15] Qian B, Huang W J, Xie D H, Ye H C, Guo A, Pan Y H, Jin Y, Xie Q Y, Jiao Q J, Zhang B Y, Ruan C, Xu T J, Zhang Y, Nie T G. Coupled maize model: a 4D maize growth model based on growing degree days. Comput Electron Agric, 2023, 212: 108124.
[16] Minorsky P V. Achieving the in silico plant. Systems biology and the future of plant biological research. Plant Physiol, 2003, 132: 404-409.
pmid: 12822566
[17] Prusinkiewicz P. Modeling plant growth and development. Curr Opin Plant Biol, 2004, 7: 79-83.
pmid: 14732445
[18] 肖亚, 李玉强. 农业人工智能综述. 数字技术与应用, 2020, 38(9): 204-205.
Xiao Y, Li Y Q. Overview of agricultural artificial intelligence. Digit Technol Appl, 2020, 38(9): 204-205 (in Chinese with English abstract).
[19] Fahlgren N, Gehan M A, Baxter I. Lights, camera, action: high-throughput plant phenotyping is ready for a close-up. Curr Opin Plant Biol, 2015, 24: 93-99.
doi: 10.1016/j.pbi.2015.02.006 pmid: 25733069
[20] Kim T H, Lee S H, Oh M M, Kim J O. Plant growth prediction based on hierarchical auto-encoder. In: International Conference on Electronics, Information, and Communication. Jeju: IEEE, 2022. pp 1-3.
[21] Kim T H, Lee S H, Kim J O. A novel shape based plant growth prediction algorithm using deep learning and spatial transformation. IEEE Access, 2022, 10: 37731-37742.
[22] Sakurai S, Uchiyama H, Shimada A, Taniguchi R. Plant growth prediction using convolutional LSTM. VISIGRAPP, 2019, 14: 105-113.
[23] 王春颖, 泮玮婷, 李祥, 刘平. 基于STLSTM的植物生长发育预测模型. 农业机械学报, 2022, 53(6): 250-258.
Wang C Y, Pan W T, Li X, Liu P. A plant growth and development prediction model based on STLSTM. Trans CSAM, 2022, 53(6): 250-258 (in Chinese with English abstract).
[24] Wang C, Pan W, Song X, Yu H, Zhu J, Liu P, Li X. Predicting plant growth and development using time-series images. Agronomy, 2022, 12: 2213.
[25] Yasrab R, Zhang J, Smyth P, Pound M P. Predicting plant growth from time-series data using deep learning. Remote Sens, 2021, 13: 331.
[26] Drees L, Junker-Frohn L V, Kierdorf J, Roscher R. Temporal prediction and evaluation of Brassica growth in the field using conditional generative adversarial networks. Comput Electron Agric, 2021, 190: 106415.
[27] Meng Y, Xu M, Yoon S, Jeong Y, Park D S. Flexible and high-quality plant growth prediction with limited data. Front Plant Sci, 2022, 13: 989304.
[28] Yang W, Guo Z, Huang C, Duan L, Chen G, Jiang N, Fang W, Feng H, Xie W, Lian X. Combining high-throughput phenotyping and genome-wide association studies to reveal natural genetic variation in rice. Nat Commun, 2014, 5: 5087.
doi: 10.1038/ncomms6087 pmid: 25295980
[29] Wang T C, Liu M Y, Zhu J Y, Tao A, Kautz J, Catanzaro B. High-resolution image synthesis and semantic manipulation with conditional GANs. Comput Vis Pattern Recognit, 2017. pp 1-14.
[30] Woo S, Park J, Lee J Y, Kweon I S. CBAM:convolutional block attention module. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y, eds. Proceedings of the European Conference on Computer Vision. Munich: Springer Cham, 2018. pp 3-19.
[31] Dowson D C, Landau B V. The fréchet distance between multivariate normal distributions. J Multivariate Anal, 1982, 12: 450-455.
[32] Korhonen J, You J. Peak signal-to-noise ratio revisited: is simple beautiful?. In: Fourth International Workshop on Quality of Multimedia Experience. Melbourne: IEEE, 2012. pp 37-38.
[33] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Proc, 2004, 13: 600-612.
[34] Wang X, Xie L, Dong C, Shan Y. Real-esrgan:training real-world blind super-resolution with pure synthetic data. In:Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021. pp 1905-1914.
[1] 贾舒涵, 何璨, 陈敏, 刘家欣, 胡伟民, 胡晋, 关亚静. 杂交水稻不同穗萌程度种子质量差异与穗萌分级研究[J]. 作物学报, 2024, 50(9): 2310-2322.
[2] 胡丽琴, 肖正午, 方升亮, 曹放波, 陈佳娜, 黄敏. 种植季节对高直链淀粉水稻品种淀粉消化特性的影响[J]. 作物学报, 2024, 50(9): 2347-2357.
[3] 刘陈, 王昆昆, 廖世鹏, 杨佳群, 丛日环, 任涛, 李小坤, 鲁剑巍. 氮肥用量对玉米-油菜和水稻-油菜轮作模式下油菜产量及氮素吸收利用的影响[J]. 作物学报, 2024, 50(8): 2067-2077.
[4] 宋志文, 赵蕾, 毕俊国, 唐清芸, 王国栋, 李玉祥. 滴灌条件下施氮量对不同氮效率水稻品种物质积累及养分吸收的影响[J]. 作物学报, 2024, 50(8): 2025-2038.
[5] 邵美红, 赵玲玲, 程楚, 程思明, 朱双兵, 翟来圆, 陈凯, 徐建龙. 水稻黄华占背景选择导入系的耐低氮筛选评价与利用[J]. 作物学报, 2024, 50(8): 1907-1919.
[6] 何丹丹, 舒亚洲, 周海连, 吴松果, 魏晓双, 杨明冲, 李波, 吴正丹, 韩世健, 杨娟, 王继斌, 王令强. OsRPTA18参与调控水稻叶片倾角的功能[J]. 作物学报, 2024, 50(8): 1934-1947.
[7] 付景, 马梦娟, 张骐飞, 段居琦, 王越涛, 王付华, 王生轩, 白涛, 尹海庆, 王亚. 干湿交替灌溉和施氮量对粳稻光合特性和氮素吸收利用的影响[J]. 作物学报, 2024, 50(7): 1787-1804.
[8] 裴法敬, 张文轩, 张晓, 王昕钰, 彭少兵, 米甲明. 长粒香型的超短生育期水稻新品系创制[J]. 作物学报, 2024, 50(7): 1684-1698.
[9] 唐清芸, 杨晶晶, 赵蕾, 宋志文, 王国栋, 李玉祥. 施氮量对滴灌水稻根系形态构型和分形特征的影响[J]. 作物学报, 2024, 50(6): 1540-1553.
[10] 张小芳, 朱琪, 华芸堰, 贾黎惠莹, 邱士优, 陈宇杰, 马涛, 丁沃娜. 水稻OsCYP22互作蛋白的筛选及验证[J]. 作物学报, 2024, 50(6): 1628-1634.
[11] 朱忠林, 文月, 周棋, 巫燕飞, 杜雪竹, 盛锋. 水稻OsCNGC10基因抗倒伏性以及抗旱性功能研究[J]. 作物学报, 2024, 50(5): 1351-1360.
[12] 胡明明, 丁峰, 彭志芸, 向开宏, 李郁, 张宇杰, 杨志远, 孙永健, 马均. 多元化种植模式下秸秆还田配合水氮管理对水稻产量形成与氮素吸收利用的影响[J]. 作物学报, 2024, 50(5): 1236-1252.
[13] 耿孝宇, 张翔, 刘洋, 左博源, 朱旺, 马唯一, 汪璐璐, 孟天瑶, 高平磊, 陈英龙, 许轲, 戴其根, 韦还和. 江苏省滨海盐碱地籼粳杂交稻产量优势形成特征[J]. 作物学报, 2024, 50(5): 1253-1270.
[14] 万应春, 班义结, 蒋钰东, 王亚欣, 刘晶晶, 刘晓晴, 程育林, 王楠, 冯萍. 水稻雄性不育突变体tpa1的表型鉴定与精细定位[J]. 作物学报, 2024, 50(5): 1104-1114.
[15] 余瑶, 王紫瑶, 周思睿, 刘鹏程, 叶亚峰, 马伯军, 刘斌美, 陈析丰. 水稻类病变突变体lms1的表型鉴定与抗病分子机制分析[J]. 作物学报, 2024, 50(4): 857-870.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
京ICP备15008789号-2