|
“Digital
Signal Processing” No. 3-2025 |
|
Image Detail Enhancement using CNN with Nested Convolution Kernels
Anisimovskiy V.V., e-mail: vanisimovsky@gmail.com
Safronova E.I., e-mail: katyasafit@gmail.com
Rychagov M.N., e-mail: michael.rychagov@gmail.com
National Research University of Electronic Technology, Zelenograd, Moscow, Russia
Keywords: image detail enhancement, fractal self-similarity, neural networks, convolutional neural network.
Abstract
We present a novel method for image detail enhancement aimed at restoring or hallucinating fine-grained natural image details while retaining well-detailed areas intact. To that end, we employ convolutional neural network trained using aligned patches from pairs of high- and low-quality images depicting the same scenery. Our training procedure includes our novel modulated retention loss which makes the learning concentrate on image areas requiring improvement, while retaining the rest.
To train our network, we propose Dual ISO/exposure training dataset collection procedure, which provides an easy and routine way of creation of the dataset consisting of pixel-aligned pairs of images of the same scene with significantly different quality of fine-grained details while having largely the same global image characteristics.
To address the problem of large-scale consistency of fine-grained details (for example, integrity of long hair strands), we propose the use of nested convolution kernels, which allows leveraging fractal self-similarity of feature maps produced from the input image. Also, nested convolution kernels allow us to drastically enlarge network receptive field without increase in network weights amount thereby providing larger visual context for image processing without the risk of overfitting.
Our experiments show clear improvement of subjective quality of fine-grained details (human hair, garment fabric) in image areas which suffered from detail degradation. Objective quality measurements (using non-reference image quality metrics, such as CPBD (Cumulative Probability of Blur Detection), BIQAA (Blind Image Quality Assessment through Anisotropy), WaDIQaM-NR (Weighted Average Deep Image QuAlity Measure for No-Reference) and BRISQUE (Blind/Referenceless Image Spatial QUality Evaluator)) show competitive performance of our method compared to the state-of-the-art image enhancement methods.
References
1. Lim B., Son S., Kim H., Nah S., Mu Lee K. Enhanced deep residual networks for single image super-resolution // Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2017, pp. 136-144.
2. Yu J., Fan Yu., Yang J., Xu N., Wang Zh., Wang X., Huang T. Wide activation for efficient and accurate image super-resolution // arXiv preprint arXiv:1808.08718. 2018.
3. Vu Th., Van Nguyen C., Pham T. X, Luu T. M., Yoo Ch. D. Fast and efficient image quality enhancement via desubpixel convolutional neural networks // Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 2018.
4. Chen Yu-Sh., Wang Yu-Ch., Kao M.-H., Chuang Yu.-Yu. Deep photo enhancer: Unpaired learning for image enhancement from photographs with gans // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018, pp. 6306-6314.
5. Ignatov A., Kobyshev N., Timofte R., Vanhoey K., Van Gool L. Dslr-quality photos on mobile devices with deep convolutional networks // Proceedings of the IEEE International Conference on Computer Vision. 2017, pp. 3277-3285.
6. Trentacoste M., Mantiuk R., Heidrich W., Dufrot F. Unsharp masking, countershading and halos: Enhancements or artifacts? // Computer Graphics Forum. 2012, vol. 31, no. 2pt3, pp. 555-564.
7. Aubry M., Paris S., Hasinoff S.W., Kautz J., Durand F. Fast local laplacian filters: Theory and applications // ACM Transactions on Graphics (TOG). 2014, vol. 33, no. 5, pp. 1-14.
8. Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y. Generative adversarial networks // arXiv preprint arXiv:1406.2661. 2014.
9. Ignatov A., Kobyshev N., Timofte R., Vanhoey K., Van Gool L. Wespe: weakly supervised photo enhancer for digital cameras // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2018, pp. 691-700.
10. Zhu J.-Y., Park T., Isola P., Efros A. Unpaired image-to-image translation using cycle-consistent adversarial networks // Proceedings of the IEEE international conference on computer vision. 2017, pp. 2223-2232.
11. Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation // Medical image computing and computer-assisted intervention--MICCAI 2015: 18th international conference. 2015, part III 18, pp. 234-241.
12. Arjovsky M., Chintala S., Bottou L. Wasserstein generative adversarial networks // International conference on machine learning, PMLR. 2017, pp. 214-223.
13. Zhang J., Pan J., Ren J., Song Y., Bao L., Lau R.W.H., Yang M.-H. Dynamic scene deblurring using spatially variant recurrent neural networks // Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, pp. 2521-2529.
14. V.V. Anisimovskiy, M.A. Penkin, E.A. Dorokhov, A.M. Gruzdev, S.S. Zavalishin, I.V. Kurilin. Slepoe ustranenie razmytiya izobrazheniya s ispol’zovaniem mgonozadachnoi svertochnoi neironnoi seti i prostranstvenno-variativnoi rekurrentnoi neironnoi seti (Blind Image Deblurring Using Multi-Task Convolutional Neural Network and Spatially Variant Recurrent Neural Network) // Trudy Mezhdunarodnoi konferentsii po kompyuternoi grafike i zreniyu "Grafikon". 2023, str. 455-462.
15. Petrova X.Y. Super-Resolution: 1. Multi-Frame-Based Approach // In: Rychagov M.N., Tolstaya E.V., Sirotenko M.Y. (eds) Smart Algorithms for Multimedia and Imaging. 2021, pp. 1-35.
16. Chernyavskiy A.S. Super-resolution: 2. Machine learning-based approach // In: Rychagov M.N., Tolstaya E.V., Sirotenko M.Y. (eds) Smart Algorithms for Multimedia and Imaging. 2021, pp. 35-58.
17. Dai T., Cai J., Zhang Y., Xia S.-T., Zhang L. Second-order attention network for single image super-resolution // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, pp. 11065-11074.
18. Liang J., Cao J., Sun G., Zhang K., Van Gool L., Timofte R. Swinir: Image restoration using swin transformer // Proceedings of the IEEE/CVF international conference on computer vision. 2021, pp. 1833-1844.
19. Zhang X., Zeng H., Guo S., Zhang L. Efficient long-range attention network for image super-resolution // European conference on computer vision. 2022, pp. 649-667.
20. Chen X., Wang X., Zhou J., Qiao Y., Dong C. Activating more pixels in image super-resolution transformer // Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023, pp. 22367-22377.
21. Hsu, Ch.-Ch., Lee Ch.-M., Chou Y.-Sh. DRCT: Saving Image Super-resolution away from Information Bottleneck // arXiv preprint arXiv:2404.00722. 2024.
22. Timofte R., Agustsson E., Van Gool L., Yang M.-H., Zhang L. Ntire 2017 challenge on single image super-resolution: Methods and results // Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2017, pp. 114-125.
23. Clevert D.-A., Unterthiner T., Hochreiter S. Fast and accurate deep network learning by exponential linear units (elus) // arXiv preprint arXiv:1511.07289. 2015.
24. Mandelbrot B.B. The fractal geometry of nature // New York. 1983.
25. Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition // arXiv preprint arXiv:1409.1556. 2014.
26. Chen T., Li M., Li Yu., Lin M., Wang N., Wang M., Xiao T., Xu B., Zhang C., Zhang Zh. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems // arXiv preprint arXiv:1512.01274. 2015.
27. Kingma D. P., Ba J. Adam: A method for stochastic optimization // arXiv preprint arXiv:1412.6980. 2014.
28. Le V., Brandt J., Lin Zh., Bourdev L., Huang Th.S. Interactive facial feature localization // European conference on computer vision. 2012, pp. 679-692.
29. Narvekar N.D., Karam L.J. A no-reference image blur metric based on the cumulative probability of blur detection (CPBD) // IEEE Transactions on Image Processing. 2011, vol. 20, no. 9, pp. 2678-2683.
30. Gabarda S., Cristobal G. Blind image quality assessment through anisotropy // JOSA A, Optical Society of America. 2007, vol. 24, no. 12, pp. B42-B51.
31. Bosse S., Maniry D., Mueller K.-R., Wiegand Th., Samek W. Deep neural networks for no-reference and full-reference image quality assessment // IEEE Transactions on image processing. 2017, vol. 27, no. 1, pp. 206-219.
32. Mittal A., Moorthy A.K., Bovik A.C. No-reference image quality assessment in the spatial domain // IEEE Transactions on image processing. 2012, vol. 21, no. 12, pp. 4695-4708.
33. Sheikh H.R., Sabir M.F., Bovik A.C. A statistical evaluation of recent full reference image quality assessment algorithms // IEEE Transactions on image processing. 2006, vol. 15, no. 11, pp. 3440-3451.
Combined technique for preliminary joint processing of radar image frames generated by circular view radar sensors of object motion control systems
I.D. Isaev, e-mail: isaevid@bmstu.ru
A.N. Savelyev, e-mail: savelyev.an@bmstu.ru
A.N. Semenov, e-mail: semenov.an@bmstu.ru
Bauman Moscow State Technical University (BMSTU), Russia, Moscow
Keywords: radar image frame, requantization, radiometric calibration, functional calibration, object resolution, contrast.
Abstract
The combined methodology based on nonlinear requantization, radiometric and functional calibration, image alignment and merging has been proposed. It is used as a preliminary joint processing of radar image frames generated by circular view radar sensors from object movement control systems. The developed technique allows for simultaneous improvement of the primary video quality and object resolution, while ensuring the requirements for timely issuance of radar information.
The choice of listed algorithms and methods for combining them into the proposed methodology is justified by following considerations. The nonlinear requantization procedure is used for informative radar image visualization on the indicator device of the operator's automation-equipped working place. The optimal values of index of a power for power transformation have been obtained, at which maximum values of the Roberts contrast and average brightness values close to the middle of the dynamic range are achieved.
The radiometric correction procedure was used as a measure to combat the nonuniform radar image brightness. It’s performed by recalculating the image pixel brightness proportional to the power of the probing signal into the space of effective radar cross-section values of observed targets, according to the radar range equation.
The resolution requirements, which usually imply falling outside limits of single radars technical characteristics, were met using the Split Bregman Algorithm (SBA). The optimal ratio values between SBA regularization parameters are substantiated, providing both effective noise suppression and increase in the image spatial resolution.
The use of the sigmoidal calibration function increases the radar image contrast and makes it possible to effectively solve the detection problem of radar target spots with intensities differing by at least 20 dB, with some expansion of target spots for more intense signals.
The need for additive mixing of radar image frames is determined by the problem of forming the resulting preprocessed joint radar image, used for the subsequent solution of primary, secondary and tertiary radar-derived information processing problems.
An analysis of the obtained results shows that the proposed methodology allows for improving the primary “raw” video quality for its visual presentation to a human operator, which is characterized by an increase in the radar image contrast by more than 2 orders and an improvement in the object spatial resolution on radar images by more than 4 times.
References
1. Vasilyev O.V., Zyabkin S.A., Ivanov A.S. et al. Multiradar concept for perspective surface movement radars // Information-measuring and control systems. 2017, vol. 15, no. 10, pp. 53-63.
2. Vasilyev O.V., Zyabkin S.A., Kalnoy M.D. et al. Multiradar surface movement radar based on surveillance X-band radar MRS-1000 // Information-measuring and control systems. 2017, vol. 15, no. 10, pp. 3-14.
3. Rymov A.I., Savelyev A.N., Semenov A.N. Experimental studies to evaluate the quality of joint information processing in a ground-based multi-position radar // Microwave & telecommunication technology. 2020, no. 1-1, pp. 535-536.
4. Shkolny L.A., Tolstov E.F., Detkov A.N. et al. Air surveillance radars, radar imagery interpretation. Moscow: Publishing house of AFEA named after prof. N.E. Zhukovsky, 2008. 531 p.: ill.
5. Guryanov M.A. Transformation of radar images dynamic range for various surfaces to be photographed // Journal of radio electronics. 2014, no. 12, p. 27.
6. Starovoitov V.V. Method of filter selection for specle-noise smoothing in SAR images // Informatics. 2015, no. 2, pp. 5-11.
7. Gonzalez R.C., Woods R.E. Digital Image Processing. 3rd Edition. Moscow: Technosphera, 2012. 1104 p.
8. Isaev I.D., Savelyev A.N., Semenov A.N. Analysis of calibration methods applied to raw radar frames received from multi-radar surface movement radars // Information-measuring and control systems. 2020, vol. 18, no. 6, pp. 51-64. DOI: 10.18127/j20700814-202006-06.
9. Isaev I.D., Savelyev A.N., Semenov A.N. Increasing radar image contrast in multi-position surface movement radars // Radar and radio communications: Proceedings of the XIV All-Russian Scientific and Technical Conference, Moscow, 2020, pp. 84-87.
10. Golub Y.I., Starovoitov F.V. Study of local assessments of contrast for digital images // System analysis and applied information science. 2019, no. 2, pp. 4-11.
11. Zhang Q. et al. Azimuth Superresolution of Forward-Looking Radar Imaging Which Relies on Linearized Bregman // IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2019, vol. 12, no. 7, pp. 2032-2043. DOI: 10.1109/JSTARS.2019.2912993.
12. Zhang Y. et al. Fast Split Bregman Based Deconvolution Algorithm for Airborne Radar Imaging // Remote Sensing. 2020, vol. 12, no. 11, p. 1747. https://doi.org/10.3390/rs1211174.
13. Huo W. et al. A Superfast Super-Resolution Method for Radar Forward-Looking Imaging // Sensors. 2021, vol. 21, no. 3, p. 817. DOI: 10.3390/s21030817.
14. Sentsov A.A., Nenashev V.A., Ivanov S.A., Turnetskaya E.L. Combining the generated radar images with a digital map of the area in on-board systems for operational monitoring of the earth's surface // Trudy MAI. 2021, no. 117. DOI: 10.34759/trd-2021-117-08.
Algorithm for positional estimation and correction of digital image resolution
Kamenskiy A.V., e-mail: andru170@mail.ru
Kuryachiy M.I., e-mail: kurtusur@mail.ru
Shmyreva A.A., e-mail: annashmyreva1805@gmail.com
Krasnoperova A.S., e-mail: alenacergeevna2@icloud.com
Federal State Autonomous Educational Institution of Higher Education – Tomsk State University of Control Systems and Radioelectronics (TUSUR), Russia, Tomsk
Keywords: Digital image processing, optical images, focal length, resolution, modulation transfer function.
Abstract
The article discusses the theoretical foundations that influence the distribution of resolution across the image field. Based on the considered theoretical foundations, the authors propose a new approach to processing large-sized images, when, taking into account the conditions for observing objects of interest and the limited characteristics of the optical system, the “full image” is divided into a number of “subimages” (areas/zones) in which their resolution is assessed. Then, based on the resolution estimates in the “subimages,” adaptive two-dimensional filtering is performed in order to level the resolution across the image field to the maximum possible, or to a specified resolution across the image field.
A software module “MIR: Multiple Resolution Measurement” was developed, which made it possible to perform multiple resolution measurements across the image field in several areas simultaneously.
According to the results of the experiments, a method of adaptive intra-frame zonal image filtering using different coefficients for each zone was proposed. It involves processes for identifying areas of poor quality and then improving them. At this stage, the process of identifying and dividing the image into areas with low resolution is based on the principles described in the section “Basics of the emergence of the technique of zone-by-zone image processing”; at further stages of development it is planned to automate this process.
The developed method makes it possible to achieve a more uniform increase in resolution and reduce the difference in quality between different areas of the image by 4 times. An approach has also been developed to determine optimal filtration coefficients depending on the treated area, which helps to increase the resolution level by 2.5-3.5 times.
References
1. Li W. et al. A fast image dehazing algorithm for highway tunnel based on artificial multi-exposure image fusion // IOP Conference Series: Materials Science and Engineering. – IOP Publishing, 2020. vol. 741. no. 1. p. 012038.
2. Ibragimova N.A., Ibragimov Z.Z. Razrabotka algoritmov tsifrovoy obrabotki signalov v zadache opticheskoy lazernoy triangulyatsii (Development of digital signal processing algorithms in the problem of optical laser triangulation) // Matritsa nauchnogo poznaniya (Matrix of scientific knowledge). 2020. No. 6. pp. 49-53.
3. Tislenko A.A., Movchan A.K., Kapustin V.V. Improving the Distance Measurement Accuracy of Active-Pulse Television Measuring Systems Using Polynomial Approximation // 2022 International Siberian Conference on Control and Communications (SIBCON). – IEEE, 2022. pp. 1-6.
4. Zaytseva E.V. Assessment of quality characteristics of television systems for the organization of communication in digital society // 2018 IEEE Communication Strategies in Digital Society Workshop (ComSDS). – IEEE, 2018. pp. 80-82.
5. Ngugi L.C., Abelwahab M., Abo-Zahhad M. Recent advances in image processing techniques for automated leaf pest and disease recognition–A review // Information processing in agriculture. 2021. vol. 8. no. 1. pp. 27-51.
6. Kornyshev N.P., Serebryakov D.A. Issledovaniye metoda selektsii izobrazheniya malorazmernogo obyekta na neodnorodnom fone (Study of the method of selecting the image of a small-sized object on a non-uniform background) // Vestnik Novgorodskogo gosudarstvennogo universiteta im. Yaroslava Mudrogo (Bulletin of the Novgorod State University. Yaroslav the Wise). 2020. vol. 121. No. 5. pp. 49-52.
7. Hou Y. et al. The state-of-the-art review on applications of intrusive sensing, image processing techniques, and machine learning methods in pavement monitoring and analysis // Engineering. 2021. vol. 7. no. 6. pp. 845-856.
8. Surendranath A., Ramana P.V. Recycled materials execution through digital image processing // Materials Today: Proceedings. 2021. vol. 46. pp. 8795-8801.
9. Vinichuk O.N., Dravitsa V.I. Razrabotka algoritmov obrabotki izobrazheniy bolshikh obyemov (Development of algorithms for processing images of large volumes) // Tsifrovaya transformatsiy (Digital transformation). 2022. vol. 28. no. 2. pp. 52-60.
10. Musihin I.D., Kapustin V.V. Video System With Automatic Tracking of the Object of Interest // 2022 IEEE International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON). – IEEE, 2022. pp. 1630-1635.
11. Zhu Z. et al. A novel fast single image dehazing algorithm based on artificial multiexposure image fusion // IEEE Transactions on Instrumentation and Measurement. 2020. vol. 70. pp. 1-23.
12. Alatwi A.M. et al. Best candidate routing algorithms integrated with minimum processing time and low blocking probability for modern parallel computing systems // Indones J Electr Eng Comput Sci. 2020. vol. 19. no. 847-54.
13. Jiang J. et al. High-speed lightweight ship detection algorithm based on YOLO-v4 for three-channels RGB SAR image // Remote Sensing. 2021. vol. 13. no. 10. p. 1909.
14. Schraivogel D. et al. High-speed fluorescence image–enabled cell sorting // Science. 2022. vol. 375. no. 6578. pp. 315-320.
15. Guzhva D.A., Sever K.O., Turulin I.I. 8-polosnyy neodnorodnyy bank bystrodeystvuyushchikh rekursivnykh filtrov s konechnoy impulsnoy kharakteristikoy dlya slukhovykh apparatov (8-band heterogeneous bank of high-speed recursive filters with finite impulse response for hearing aids) // Izvestiya YuFU. Tekhnicheskiye nauki (Izvestia SFU. Technical Sciences). 2023. no. 6. pp. 34-44.
16. Konanykhin A.Yu., Konanykhina T.N., Panishchev V.S. Metody uluchsheniya vydelennoy oblasti izobrazheniya pri bystrodeystvuyushchey obrabotke simvolnoy informatsii (Methods for improving the selected image area during high-speed processing of symbolic information) // Izvestiya Yugo-Zapadnogo gosudarstvennogo universiteta. Seriya: Upravleniye. vychislitelnaya tekhnika. informatika. Meditsinskoye priborostroyeniye (News of the South-Western State University. Series: Management, computer technology, computer science. Medical instrumentation). 2023. vol. 11. no. 4. pp. 106-119.
17. Kamenskiy A.V. High-speed recursive-separable image processing filters. Computer Optics 2022; 46(4): 659-665. DOI: 10.18287/2412-6179-CO-1063.
18. Kamenskiy A.V., Kuryachiy M.I., Krasnoperova A.S., Ilyin Y.V., Akaeva Ò.Ì., Boyarkin S.E. High-speed recursive-separable image processing filters with variable scanning aperture sizes. Computer Optics 2023; 47(4): 605-613. DOI: 10.18287/2412-6179-CO-1240.
19. Knyazev D.V. Obzor nekotorykh algoritmov segmentatsii izobrazheniy (Review of some image segmentation algorithms) // Izvestiya Tulskogo gosudarstvennogo universiteta. Tekhnicheskiye nauki (News of Tula State University. Technical Sciences). 2021. no. 10. pp. 393-400.
20. Nguyen A. T. Segmentatsiya izobrazheniy s shirokim dinamicheskim diapazonom metodom narashchivaniya oblastey (Segmentation of images with a wide dynamic range using the method of growing areas).: dis. – BSUIR, 2021.
21. Ge S. et al. Expressive text-to-image generation with rich text // Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. pp. 7545-7556.
22. Alekseychuk A.S., Mukin Yu.D. Primeneniye svertochnykh neyronnykh setey v zadache ustraneniya teney s fotografiy (Application of convolutional neural networks in the problem of eliminating shadows from photographs) // Modelirovaniye i analiz dannykh (Modeling and data analysis). 2024. vol. 14. no. 1. pp. 41-51.
23. Pan Z., Cai J., Zhuang B. Fast vision transformers with hilo attention // Advances in Neural Information Processing Systems. 2022. vol. 35. pp. 14541-14554.
24. Sagaidak D.A. Modeli i algoritmy realizatsii sistemnogo podkhoda k organizatsii raspredelennoy peredachi videodannykh (Models and algorithms for implementing a system approach to organizing distributed video data transmission): Dis. cand. those. Sciences: 2.3.1 D. A. Sagaidak. – M., 2025. p. 181.
25. Mitrofanov S.S. Optical measuring instruments and devices. – 2018.
26. Iofis E. A. Photocinema technology. Encyclopedia. – 1981.
27. Gordiychuk O. F., Pell V. G. Section III. Cinematography lenses // Directory of cinematographer / NN Zherdetskaya. – M.: Art. – 1979.
28. GOST 7427-76. Geometric optics. Terms, definitions and letter designations.
29. Llozzi M. History of physics. – 1970.
30. Gruzman I.S. et al. Tsifrovaya obrabotka izobrazheniy v informatsionnykh sistemakh (Digital image processing in information systems) // Novosibirsk: Izd-vo NGTU (Novosibirsk: NSTU Publishing House). 2002. vol. 352. p. 6.
31. Fomin A.V. General course in photography: Textbook for technical schools, 3rd ed. – M. – 1987.
32. Slyusarev G. G. Methods for calculating optical systems. – 1969.
33. Certificate of state registration of a computer program No. 2019660074. MIR: Multiple measurement of resolution. Author: A.V. Kamensky. Application No. 2019618804. Date of receipt July 17, 2019. Registered in the Register of Computer Programs on July 30, 2019.
34. Rudnikovich A.S., Shipunova K.V. ISO 12233 i programmnyye sredstva izmereniya chastotno-kontrastnykh kharakteristik izobrazheniy (ISO 12233 and software for measuring frequency-contrast characteristics of images) // Trudy Mezhdunarodnoy konferentsii po kompyuternoy grafike i zreniyu "Grafikon" (Proceedings of the International Conference on Computer Graphics and Vision "Graphicon"). 2018. no. 28. pp. 48-51.
35. Certificate of state registration of a computer program No. 2019660075. SFI: Line-by-line filtering of images. Author: A.V. Kamensky. Application No. 2019618808. Date of receipt July 17, 2019. Registered in the Register of Computer Programs on July 30, 2019.
Application of integral characteristics of energy spectra for object detection in images
A.V. Bogoslovsky, e-mail: p-digim@mail.ru
S.V. Vasilyev, e-mail: stanislav-vas1986@mail.ru
I.V. Zhigulina, e-mail: ira_zhigulina@mail.ru
MESC AF «N.E. Zhukovsky and Y.A. Gagarin Air Force Academy» (Voronezh)
Keywords: vector field, flow, circulation, energy spectrum, phase-energy spectrum, detection.
Abstract
The approaches to object detection that are currently known are characterized by a wide variety, but they also have some drawbacks that limit their unconditional application. This is primarily due to the specific features of the background and target environment that are reflected in the recorded images. When the background is relatively uniform, correlation methods, linear prediction methods, adaptive linear filtering, and others are used. However, in images captured in challenging conditions, objects may be almost indistinguishable from the background, making it difficult to detect them automatically.
An approach to object detection in digital images based on the analysis of two vector fields – the gradient of the energy spectrum (ES) and the phase-energy spectrum (PES) of the image, which combines the advantages of the energy and phase-frequency spatial spectra of the signal – is proposed.
Form a field of circulation values in the first frequency quadrant of the phase-energy spectrum of a set of images obtained from the original image by placing a special scanner object with a specified aperture and brightness and moving it sequentially across the entire image field. Search for all local extremums in the formed field, the coordinates of which carry information about the position of the geometric centers of the detected objects.
The proposed approach to object detection is characterized by a sufficiently high sensitivity and algorithmic speed, which allows it to be used in real-time image registration and processing systems, particularly in space monitoring systems where special attention is paid to the peripheral part of the sensor field of view due to the need for immediate action (airspace protection), as well as in the form of a pre-processing contour for highlighting all low-contrast areas that may potentially be objects of interest.
References
1. Sitnikov V.V., Lyuminarskii V.V, Korobeinikov A.V. Obzor metodov raspoznavaniya obektov, ispolzuemikh v sistemakh mashinnogo zreniya // Vestnik IzhGTU imeni M.T. Kalashnikova. 2018. T. 21. ¹ 4. S. 222–229.
2. Krasnov F.V., Butorin A.V., Sitnikov A.N. Obzor podkhodov k analizu prostranstvennikh izobrazhenii visokogo razresheniya dlya primeneniya v geofizike // Intellektualnie sistemi. 2019. T. 6. ¹ 1. 127–143.
3. Subbotin V.V., Andreev S.M. Analiz metodov kompyuternogo zreniya dlya raspoznavaniya obektov v promishlennosti // Avtomatizirovannie tekhnologii i proizvodstva. 2024. ¹ 2. S. 22–26.
4. Sirota A.A., Mitrofanova Ye.Yu., Milovanova A.I. Analiz algoritmov poiska obektov na izobrazheniyakh s ispolzovaniem razlichnikh modifikatsii svertochnikh neironnikh setei // Vestnik VGU. Seriya: Sistemnii analiz i informatsionnie tekhnologii. 2019. ¹ 3. S. 123–137.
5. Yang F., Sun Q., Jin H., Zhou Z. Superpixel segmentation with fully convolutional networks // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA: IEEE, 2020. P. 13961–13970. https://doi.org/10.1109/CVPR42600.2020.01398.
6. Khimenko V.I., Okhtilev M.Yu., Klyucharev A.A., Matyash V.A. Analiz informativnikh priznakov v zadachakh obrabotki dannikh aerokosmicheskogo monitoringa // Obrabotka informatsii i upravlenie. 2017. ¹ 2. S. 2–12. DOI: 10.15217/issn1684-8853.2017.2.2.
7. Uijlings J.R.R., Van de Sande K.E.A., Gevers T. & Smeulders A.W.M. Selective search for object recognition // International journal of computer vision. 2013. 104(2), P. 154 171.
8. Alpatov B.A., Babayan P.V., Yershov M.D. Podkhodi k obnaruzheniyu i otsenke parametrov dvizhushchikhsya obektov na videoposledovatelnosti primenitelno k transportnoi analitike // Kompyuternaya optika. 2020. ¹ 5. S. 746–756.
9. Mareev A.V., Orlov A.A., Rizhkova M.N. Metodi lokalizatsii obektov v videopotoke // Radiotekhnicheskie i telekommunikatsionnie sistemi. 2021. ¹ 3. S. 48–60.
10. Zhigulina I.V. Energeticheskie kharakteristiki izobrazhenii i videoposledovatelnostei // Televidenie: peredacha i obrabotka izobrazhenii: materiali 13-i Mezhdunarod. konf. SPb: SPb. GEU «LETI» im. V.I. Ulyanova (Lenina). 2016. S. 128–131.
11. Borisenko A.I., Tarapov I.E. Vektornii analiz i nachala tenzornogo ischisleniya: Ucheb. posobie dlya vuzov / A.I. Borisenko. Kharkov: Izd-vo Khark. un-ta, 1959. 238 s.
12. Bogoslovskii A.V., Zhigulina I.V., Sukharev V.A. Vektornoe pole fazoenergeticheskogo spektra izobrazheniya i videoposledovatelnosti // Radiotekhnika. 2018. ¹ 11. S. 11–16
13. Bogoslovskii A.V., Sukharev V.A., Zhigulina I.V., Pantyukhin M.A. Vektornie polya, porozhdaemie preobrazovaniem Fure videosignalov izobrazhenii // Radiotekhnika. 2021. ¹ 7. S. 127–139.
14. Vasilev S.V., Bogoslovskii A.V., Zhigulina I.V. Analiz dvizheniya grupp obektov na osnove fazoenergeticheskogo spektra videoposledovatelnosti // Tsifrovaya obrabotka signalov. 2024. ¹ 2. S. 19–26.
15. Goldfain I.A. Vektornii analiz i teoriya polya / Pod red. i s predisl. R.S. Gutera. M.: Fizmatlit. 1962. 132 s.
Interference-resistant processing of multichannel signals using empirical mode decomposition
Kramm M.N., e-mail: krammmn@mail.ru
Bodin A.Yu., e-mail: bodin98@mail.ru
National Research University «Moscow Power Engineering Institute» (MPEI), Russia, Mosñow
Keywords: interference-resistant processing of electrocardiosignals, empirical mode decomposition, nonlinear threshold filtering, Hilbert-Huang transform.
Abstract
The work is devoted to digital filtering of interference in multichannel non-stationary signals using empirical mode decomposition (EMD). In the tasks of transmission of multi-channel signals, an important role is played by interference-resistant processing of selected portions of accepted signals (frames) carrying important information. This is the case with multi-channel telemetry systems that transmit information about the state of a complex object using an ensemble of sensors for continuous monitoring. An example is the information multichannel cardiology systems, in which the electrocardiosignals are registered from electrodes placed evenly on the torso surface. The problem is to suppress both high-frequency interference (noise, network interference) and low-frequency interference (especially drift of the electrode potential).
Algorithms for nonlinear non-negative threshold filtering are proposed that implement the suppression of interference concentrated in the first empirical modes (high-pass filtering), as well as an algorithm for the case when the interference is contained in the remainder of the decomposition and in the last empirical mode (low-pass filtering).
To evaluate the filtration efficiency, analysis of the median frequencies and amplitudes of empirical modes calculated by the Gilbert-Huang conversion is used. We represent estimates of filtration error using the example of electrocardiosignals.
The method of empirical model decomposition can be used as a basis for filtering interferences in the registration of non-linear nonstationary processes. This method, with a limited number of EMs used, is memory-efficient and effective when the bulk of the interference energy is contained in the first EMs or in the remainder and the last EM.
References
1. Titomir L.I., Trunov V.G., Aidu E.A.I. Noninvasive Electrocardiotopography. Moscow: Nauka, 2003. – 198 p.
2. Polyakova I.P. Surface ECG Mapping as a Method for Diagnosing Cardiac Rhythm Disorders. Chapter 6 in the monograph Noninvasive Diagnostics in Clinical Arrhythmology. Moscow: Meditsina, 2009, pp. 157–175
3. New Methods of Electrocardiography. Edited by S.V. Grachev, G.G. Ivanov, and A.L. Syrkin. Moscow: Tekhnosfera, 2007. – 552 p.
4. Bokeria L.A., Revishvili F.Sh., Kalinin F.V., Kalinin V.V., Lyakhina O.S., Fetisova E.A. Hardware–Software System for Noninvasive Electrophysiological Cardiac Studies Based on the Inverse Electrocardiography Problem. Medical Engineering, 2009, No. 6, pp. 1–7.
5. Li L., Camps J., Rodriguez B., Gra V. Solving the Inverse Problem of Electrocardiography for Cardiac Digital Twins: A Survey. IEEE Reviews in Biomedical Engineering, July 2024, pp. 1–19.
6. Kramm M.N., Chong T.L.N., Bodin A.Yu., Bodin O.N., Zhikhareva G.V. Algorithm for Processing Electrocardiographic Signals in a Multielectrode Electrocardiological Screening System for Visualizing Epicardial Electrical Potential. Medical Engineering, 2023, No. 5(341), pp. 13–17.
7. Bodin A.Yu., Kramm M.N., Krivonogov L.Yu., Serzhantova N.A., Chong T.L.N. Classification of Electrocardiographic Noise and Development of a Method for Segmenting Electrocardiographic Signals. Measurement. Monitoring. Control., 2023, No. 4, pp. 64–71.
8. Baskakov S.I. Radio Engineering Circuits and Signals. Moscow: Vysshaya Shkola, 1988.
9. Smolentsev N.K. Fundamentals of Wavelet Theory. Wavelets in MATLAB. Moscow: DMK Press, 2014. – 628 p.
10. Huang N.E., et al. The Empirical Mode Decomposition and the Hilbert Spectrum for Nonlinear and Non-Stationary Time Series Analysis. Proceedings of the Royal Society of London A, 1998, vol. 454, pp. 903–995.
11. Baevsky R.M., Ivanov G.G., Chireikin L.V., et al. Analysis of Heart Rate Variability Using Various Electrocardiographic Systems (Methodological Guidelines). Vestnik Arritmologii, 2001, No. 24, pp. 65–87.
12. Huang N.E., Shen S.S.P. Hilbert–Huang Transform and Its Applications (2nd ed.). Singapore: World Scientific, 2014. – 400 p.
13. Krivonogov L.Yu. Analysis and Processing of Empirical Modes for Noise Suppression in Electrocardiographic Signals. Izvestiya SFedU. Technical Sciences. Thematic Issue “Medical Information Systems”, Taganrog: Publishing House of TTI SFedU, 2012, No. 9 (134), pp. 119–124.
14. Breiman L. Better Subset Regression Using the Nonnegative Garrote. Technometrics, 1995, vol. 37, no. 4, pp. 373–384.
Adaptive polynomial filtering algorithms in the frequency domain
M.A. Shcherbakov, e-mail: mashcherbakov@yandex.ru
Penza State University (PSU), Russia, Penza
Keywords: adaptive nonlinear filtering, polynomial filters, spectral transforms, discrete Volterra series.
Abstract
Algorithms for adapting polynomial filters in the frequency domain have been developed. These algorithms utilize block processing of input data to smooth the convergence rate and reduce the level of random fluctuations in filter coefficients. At the same time, the solution obtained during the adaptation process does not guarantee a minimum mean square error in the traditional sense, as these algorithms operate with cyclic convolutions and correlations.
To eliminate this effect, a modification of the block algorithm in the frequency domain is proposed, based on partitioning the data into intersecting blocks. An estimate of the range of adaptation parameters is provided, guaranteeing the convergence of the adaptation process. It is shown that with a sequential organization of the adaptation process, the permissible range of adaptation parameter variation can be increased, enabling higher convergence rates.
A sequential adaptation algorithm in the frequency domain is proposed, using an exponential weighting procedure to estimate the upper bound of the adaptation parameters. Recursive Newtonian frequency-domain adaptation algorithms are also considered, allowing for the algorithm's convergence rate to be adjusted both for individual frequencies and for individual nonlinear filter components.
To reduce computational costs, it is proposed to exploit the symmetry properties of the filter's frequency responses, as well as a priori information about the input signal. This approach is demonstrated using a third-order polynomial filter for a class of input signals with a normal probability distribution.
References
1. Shcherbakov M.A. Algoritmy` adaptivnoj polinomial`noj fil`traciya vo vremennoj oblasti (Adaptive polynomial filtering algorithm in the time domain) // Cifrovaja obrabotka signalov (Digital signal processing). 2024, no. 1, pp. 21–28.
2. Adaptive filters / Ed. C.F.N. Cowan and P.M. Grant. Publisher: Prentice-Hall, Inc., Englewood Cliffs, NJ. 1985. 308 p.
3. Widrow B., D. Stearns S. D. Adaptive signal processing. Prentice-Hall. 1985. 474 p.
4. Dzhigan V.I. Adaptivnaya fil`traciya signalov: teoriya i algoritmy` (Adaptive signal filtering: theory and algorithms.). M.: Tekhnosfera. 2013. 528 p.
5. Narayan, S. S., & Peterson, A. M. (1981). Frequency domain least-mean-square algorithm // Proceedings of the IEEE. 1981, vol. 69, no.1, pp. 24–126.
6. Scherbakov, M.A. Designing pareto optimal nonlinear filters for image processing. Autom. Remote Control. 2010, vol.71, pp. 339–351.
7. Shcherbakov M.A., Krevchik V.D., Sazonov V.V. An algebraic approach to implementation of generalized polynomial filters // 2015 International Siberian Conference on Control and Communica-tions. (SIBCON). DOI: 10.1109/SIBCON.2015.7147293.
8. Mansour D., Gray A.H. Frequency domain non-linear adaptive filter // Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, Atlanta. 1981, pp. 550–553.
9. Marmarelis V., Marmarelis M. Analysis of physiological systems. The White-Noise Approach. New York: Plenum Press. 1978. 488 p.
Development and research of primary speech codecs in multi-stage decomposi-tion based on modifications of the Khurgin-Yakovlev algorithm
V. T. Dmitriev, e-mail: vol77@rambler.ru
Vu Hoang Son, e-mail: vuhoangson.adaf@gmail.com
The Ryazan State Radio Engineering University (RSREU), Russia, Ryazan
Keywords: modification of the Khurgin-Yakovlev algorithm, multi-stage decomposition, primary encoding algorithm, noise immunity, V. A. Kotelnikov's theorem.
Abstract
The article presents the design, implementation, and evaluation of primary speech codecs using a multi-stage decomposition approach based on a modified Khurgin–Yakovlev algorithm. The core objective is to enhance noise immunity and improve speech signal reconstruction accuracy under constrained bandwidth and non-ideal transmission environments. The proposed algorithm utilizes a phase-shifted representation of decimated signal samples and their derivatives, effectively reducing the required sampling rate while preserving information fidelity. This represents a fundamental departure from traditional approaches grounded in Kotelnikov’s sampling theorem, which assumes ideal channel conditions and full-bandwidth signal reconstruction.
A generalized system architecture is developed, enabling recursive decomposition with configurable numbers of channels (M) and stages (P). The synthesis filters are optimized in both amplitude and phase response domains, with approximation strategies employed to minimize reconstruction error. The algorithm supports scalable implementation from two to four channels and up to three decomposition stages, allowing trade-offs between computational complexity and speech quality.
To validate the effectiveness of the proposed system, extensive simulation studies were conducted using standardized speech datasets following ITU-T P.800 recommendations. Speech signals were encoded using three common codecs — MMBE (1.2 kbps), G.723.1 (6.3 kbps), and G.726 (24 kbps) — and tested under various bit error rates (from 0.1% to 5%) to simulate real-world communication channels. Evaluation metrics included Signal-to-Noise Ratio (SNR), Mean Squared Error (MSE), and ViSQOL—an objective speech quality metric aligned with the MOS scale.
The results show that the modified Khurgin–Yakovlev algorithm based multi-stage decomposition significantly improves speech quality in both ideal and noisy conditions. For the low-bitrate MMBE codec, up to 1.96 MOS gain was achieved; for G.723.1 and G.726, the improvements were up to 2.17 and 1.34 MOS, respectively, over classical sampling-based methods. Additionally, systems configured with higher values of M and P exhibited greater resilience to channel errors, with performance remaining acceptable even under severe impairments.
Analysis confirms that the Khurgin–Yakovlev-based multi-stage system not only approaches optimal coding efficiency in ideal channels but also maintains high quality in significantly degraded communication conditions. The achieved coding gain, especially at low and medium bitrates, notably exceeds that of traditional codecs using classical sampling theory.
References
1. Khurgin Ya.I., Yakovlev V.P. Metody teorii celykh funktsiy v radiofizike, teorii svyazi i optike (Methods of the Theory of Entire Functions in Radiophysics, Communication Theory, and Optics). M.: Gosudarstvennoe izdatel'stvo fiziko-matematicheskoy literatury, 1962. 220 p.
2. Dmitriev V.T., Luk'yanov D.I. Algoritm maskirovaniya na osnove predstavleniya Khurgina–Yakovleva s ispol'zovaniem proizvodnykh vtorogo i tret'ego poryadkov // Vestnik RSREU, 2012, No.4, pp. 13–17.
3. Kirillov S.N., Dmitriev V.T., Krysaev D.E., Popov S.S. Issledovanie kachestva peredaemoy rechevoy informatsii pri razlichnom sochetanii algoritmov kodirovaniya istochnika i kanala svyazi v usloviyakh deystviya pomekh // Vestnik RSREU, 2008, No.1 (Issue 23), pp. 53–56.
4. Dmitriev V.T., Kharlanova E.A. Algoritm kodirovaniya rechevykh signalov na osnove predstavleniya Khurgina–Yakovleva i veyvlet-paketnogo razlozheniya v sistemakh komp'yuternoy telefonii // Vestnik RSREU, 2010, No.1 (Issue 31), pp. 98–101.
5. Dmitriev V.T., Vu Hoang Son Primenenie trekhkanal'noy modifikatsii algoritma Khurgina–Yakovleva v algoritmakh pervichnogo kodirovaniya rechevykh signalov // Vestnik RSREU, 2024, No.88, pp. 3–14.
6. Andreev V.G., Dmitriev V.T. Algoritm sovmestnoy realizatsii pervichnogo kodeka i maskiratora rechevykh signalov s vozmozhnost'yu zashchity fonogramm ot falsifikatsiy // Vestnik RSREU, 2023, No.84, pp. 66–76.
7. Kirillov S.N., Dmitriev V.T. Adaptive Primary Speech Signals Codecs for Software-Configured Radio Systems. // 2020 1st International Conference on Problems of Informatics, Electronics, and Radio Engineering (PIERE). IEEE, DOI: 10.1109/PIERE51041.2020.9314648. pp. 32–38.
8. Dmitriev V.T., Konstantinova D.S. Algoritm kompleksnoy otsenki kachestva rechi v kanale svyazi // Vestnik RSREU, 2016, No.56, pp. 42–47.
9. Bakhurin S.A., Dmitriev V.T. Issledovanie tochnosti algoritmov otsenki otschetov proizvodnoy v radiotekhnicheskikh ustroystvakh // Vestnik RSREU, 2004, No.13, pp. 32–35.
10. Dmitriev V.T., Lantratov S.Yu. Adaptivnyy algoritm kodirovaniya na osnove kodeka CELP i modifikatsii algoritma Khurgina–Yakovleva // Vestnik RSREU, 2024, No.88, pp. 21–30.
11. Dmitriev V.T. Adaptatsiya kodekov rechevykh signalov na osnove teoremy V.A. Kotel'nikova i modifikatsii algoritma Khurgina–Yakovleva k shumam v kanale svyazi // Tsifrovaya Obrabotka Signalov (Digital Signal Processing), 2023, No.2, pp. 55–60.
12. Kirillov S.N., Dmitriev V.T. Modifikatsiya band vokodera na osnove predstavleniya Khurgina–Yakovleva i algoritma Fienupa // In: 14th Int. Sci-Tech Conf. on Actual Problems of Electronic Instrument Engineering (APEIE) – 2018, pp. 192–196.
13. Kirillov S.N., Dmitriev V.T., Kartavenko Ya.O. Algoritm ob"ektivnoy otsenki kachestva dekodirovannogo rechevogo signala na osnove izmeneniya spektral'noy dinamiki kriticheskikh polos spektra // Vestnik RSREU, 2011, No.3 (Issue 37), pp. 3–7.
14. Kirillov S.N., Dmitriev V.T. Ustoychivost' pervichnykh kodekov rechevykh signalov na osnove predstavleniya Khurgina–Yakovleva k deystviyu akusticheskikh shumov // Vestnik RSREU, 2019, No.3, pp. 17–25.
15. Dmitriev V.T., Smirnov M.S. Issledovanie pomekhoustoychivoy i zashchishchennoy sistemy peredachi rechevykh signalov na osnove predstavleniya Khurgina–Yakovleva // Vestnik RSREU, 2022, No.82, pp. 27–37.
16. Dmitriev V.T. Pomekhoustoychivost' kodekov rechi na osnove algoritma Khurgina–Yakovleva // Vestnik RGRTU, 2003, No.12, pp. 133–136.
If you have any question please write: info@dspa.ru