|
“Digital
Signal Processing” No. 3-2017 |
|
“Digital economy”, world practice of the development of highly efficient digital systems for TV and radio broadcasting and the problems of its implementation in Russia
A.V. Dvorkovich, e-mail: dvork-alex@yandex.ru
V.P. Dvorkovich, e-mail: v.dvorkovich@mail.ru
M.A. Sedova, e-mail: sedovamarina2017@yandex.ru
Moscow Institute of Physics and Technology (State University) (MIPT), Russia, Dolgoprudniy
Keywords: digital economy, digital TV and radio broadcasting, videoconferencing, multimedia encoding, quality of service QoS, quality of experience QoE.
Abstract
Digital information technologies under wide use and implementation should provide the high efficiency of «digital economy» utilization, that actualizes the modern trends of TV and radio broadcasting and communication.
The variants of highly efficient digital TV and sound broadcasting and videoconferencing development based on the latest advances in multimedia coding and transmission technology have been analyzed.
Activities on the development of new television broadcasting system variants are carried out through the world. They provide the increase of the quality of transmitted information and viewing perception. These activities include such technologies as ultrahigh definition television (UHDTV), high dynamic range (HDR), high framerate (HFR), immersive multichannel sound accompaniment, etc.
Implementation of the digital terrestrial sound broadcasting in I-III VHF bands is going on gradually. This broadcasting provides the transmission of multimedia information, including both stereo sound with various quality and static and dynamic video signals, other additional digital data.
The most important problem of increasing the efficiency of modern developments is the analysis of the quality of transmitted and reproduced multimedia content. So far, there are no generally unified methods for assessing the quality of service (QoS) and quality of experience (QoE). In this regard, the development of intellectual methods of QoE assessment with the use of machine learning methods to automatically assess the quality of perception on the basis of objective parameters. Sets of the analyzed indicators, ways of their reception and approaches to the analysis are considered.
The results of the new broadcasting systems development allow to state positively about the possibilities of principal change of the information presentation quality. Broadcasting systems don’t realize such functional possibilities up to now. Considering these world achievements, it should be mentioned that they are not deployed in Russia. Insufficient attention is payed to the implementation of digital TV technologies of standard and high definition, deployment of national broadcasting systems that provide sharp increase of radio spectrum usage efficiency, etc.
References
1. “About the strategy of information society development in Russian Federation for 2017 – 2030”. The decree ¹203 of the Russian Federation President Putin V.V. from 9 May 2017 (in Russian).
2. The Program “Digital economy of Russian Federation” approved by the order ¹ 1632-p of Russian Federation Government from 20 July 2017 (in Russian).
3. “About the strategy of economic safety of Russian Federation for the period up to 2030”. The decree ¹208 of the Russian Federation President Putin V.V. from 9 May 2017 (in Russian).
4. Leites L.S. Ocherki istorii otechestvennogo televideniya (Sketch-book of history of national television). 2-nd edition. Moscow: Ostankino, 2017, 223 p. (in Russian).
5. Leites L.S. Televidenie Rossii. Znamenatel’nye fakty, nauchno-tkhnicheskaya bibliografiya, biografii izvestnykh uchenykh I spetsialistov tekhniki televideniya (Television of Russia. Remarkable facts, scientific-technical bibliography, biographs of familiar scientists and specialists of television technique). Handbook. Moscow: «Perviy tom» publisher, 2017, 580 p. (in Russian).
6. Report ITU-R BT.2380-0. Television colorimetry elements (07/2015).
7. Report ITU-R BT.2390-1. High dynamic range television for production and international programme exchange (10/2016).
8. Report ITU-R BS.2159-7. Multichannel sound technology in home and broadcasting applications (02/2015).
9. DIGITAG Digital Television Action Group. DTT Technology Information. DigiTAG - The Digital Terrestrial Television Action Group, 2015.
10. DIGITAG Digital Television Action Group. Guide to Digital Switchover. Focus Africa and Asia. DigiTAG – The Digital Terrestrial Television Action Group, 2013.
11. Framing Structure, Channel Coding and Modulation for Digital Television Terrestrial Broadcasting System (in Chinese). Chinese National Standard GB20600-2006.
12. Frame Structure, Channel Coding and Modulation for Digital Television/Terrestrial Multimedia Broadcasting-Advanced (in Chinese). Tsinghua University. National Engineering Lab. for DTV (Beijing) (DTVNEL).
13. Dvorkovich V.P., Dvorkovich A.V. Tsifrovye videoinformatsionnye sistemy (teoriya i praktika) (Digital video information systems (theory and practice)). Moscow: Tekhnosfera, 2012. 1007 p. (in Russian).
14. Dvorkovich V.P., Dvorkovich A.V. Izmereniya v videoinformatsionnykh sistemakh (teoriya i praktika) (Measurements in video information systems (theory and practice)). Moscow: Tekhnosfera, 2015. 873 p. (in Russian).
15. Dvorkovich V.P., Dvorkovich A.V. Okonnye funktsii dlya garmonicheskogo analiza signalov (Window functions for harmonic analysis of signals). Second edition, revised and com-plemented. Moscow: Tekhnosfera, 2016. 208 p. (in Russian).
16. Report ITU-R BT. 2295-1. Digital terrestrial broadcasting systems (07/2015).
17. Dvorkovich A.V., Dvorkovich V.P., Zubarev Yu.B, Sokolov A.Yu., Chernov Yu.A. Sposob translyatsii informatsionnogo televideniya (Method for transmission of information televi-sion). Russian Federation Patent ¹ 2219676, priority from 08.11.2000 (in Russian).
18. Dvorkovich A.V., Dvorkovich V.P., Irtyuga V.A. Sposob mobil’nogo uzkopolosnogo tsifrovogo mul’timediynogo radioveshaniya (Method for mobile narrowband digital multimedia radio broadcasting). Russian Federation Patent ¹ 2441321, 27.01.2012, priority from 26.07.2010 (in Russian).
19. GOST R 54309-2011. Audiovizual'naya informacionnaya sistema real'nogo vremeni (RAVIS). Processy formirovaniya kadrovoj struktury, kanal'nogo kodirovaniya i modulyacii dlya sistemy cifrovogo nazemnogo uzkopolosnogo radioveshchaniya v OVCH-diapazone. Tekhnicheskie usloviya (Real-time audiovisual information system (RAVIS). The processes of frame structure formation, channel coding and modulation for digital terrestrial narrowband broadcasting system in VHF band. Technical specification) (in Russian).
20. GOST R 55686-2013. Audiovizual'naya informacionnaya sistema real'nogo vremeni (RAVIS). Cifrovoj modulyator. Osnovnye parametry i tekhnicheskie trebovaniya (Real-time audiovisual information system (RAVIS). Digital modulator. Main parameters and technical reqiurements) (in Russian).
21. GOST R 55687-2013. Audiovizual'naya informacionnaya sistema real'nogo vremeni (RAVIS). Kontrol'nyj radiopriemnik. Obshchie tekhnicheskie trebovaniya (Real-time audiovisual information system (RAVIS). Test receiver. General technical requirements) (in Russian).
22. GOST R 55688-2013. Audiovizual'naya informacionnaya sistema real'nogo vremeni (RAVIS). Formirovatel' kontenta. Struktura i protokoly peredachi dannyh (Real-time audiovisual information system (RAVIS). Content composer. Structure and protocols for data transmission) (in Russian).
23. GOST R 55689-2013. Audiovizual'naya informacionnaya sistema real'nogo vremeni (RAVIS). Normy i metody metrologicheskogo obespecheniya (Real-time audiovisual information system (RAVIS). Norms and methods for metrological support) (in Russian).
24. Report ITU-R BT.2049-7. Broadcasting of multimedia and data applications for mobile reception (02/2016).
25. Report ITU-R BS.2214-2. Planning parameters for terrestrial digital sound broadcasting systems in VHF bands (10/2016).
26. Certificate of approval of measurement tool type RU.C.35.002.A ¹ 57904. Kompleksy izmeritel'nye televizionnye KI-TVM (TV measurement sets KI-TVM) (in Russian).
27. Certificate of approval of measurement tool type RU.C.35.002.A ¹ 57905. Kompleksy izmeritel'nye televizionnye KI-TVM-E (TV measurement sets KI-TVM-E) (in Russian).
28. Certificate of approval of measurement tool type RU.C.35.002.A ¹ 57903. Kompleksy izmeritel'nye cifrovogo veshchatel'nogo televideniya KI-CVT (Digital broadcast TV measurement sets KI-CVT) (in Russian).
29. Mobile telemedicine complex MTMC. SAD-COM Ltd. advertising booklet, 2017 (in Russian).
Video background reconstruction by non-parametric motion completion and frame-by-frame refinement
A.A. Bokov, e-mail: abokov@graphics.cs.msu.ru
D.S. Vatolin , e-mail: dmitriy@graphics.cs.msu.ru
Lomonosov Moscow State University, Russia, Moscow
Keywords: background reconstruction, motion completion, video processing.
Abstract
In this paper we introduce a novel algorithm for video background reconstruction, which is an important problem in video processing that has numerous applications, including video restoration, rig removal and occlusion filling in virtual view synthesis. In recently proposed algorithms, background reconstruction is usually framed as an optimization problem over the whole spatiotemporal domain, which is quite expensive to solve and leads to high computational complexity in related background reconstruction approaches. We address this issue by replacing global spatiotemporal optimization with a multipass background reconstruction pipeline that includes a general non-parametric method for motion completion and a frame-by-frame variational refinement step to alleviate the effect of error accumulation. We also propose a novel Poisson blending modification to perform illumination adjustment in our framework, as well as a separable approximation technique to decrease its computational complexity. As a result, the proposed algorithm can handle a wide variety of scenes captured with a free-moving camera and maintain the level of quality that is comparable to the current state-of-the-art approaches, while achieving much faster processing speed.
To verify the performance of the proposed background reconstruction algorithm we conduct an experimental comparison with six existing algorithms on seven test video sequences using two quality metrics that have demonstrated high correlation with human perception of background reconstruction quality. According to both metrics, the proposed algorithm demonstrates the best quality on average. Moreover, we provide several examples that illustrate our background reconstruction results and compares them with the results of Huang et al. [8].
References
1. J. Liu, G. Zhai, X. Yang, B. Yang, and L. Chen, “Spatial error concealment with an adaptive linear predictor,” IEEE Transactions on Circuits and Systems for Video Technology, pp. 353–366, 2015.
2. Y. Matsushita, E. Ofek, W. Ge, X. Tang, and H. Y. Shum, “Full-frame video stabilization with motion inpainting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1150-1163, 2006.
3. A. Sobral and A. Vacavant, “A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos,” Computer Vision and Image Understanding, pp. 4-21, 2014.
4. A. Newson, A. Almansa, M. Fradet, Y. Gousseau, and P. Perez, “Video inpainting of complex scenes,” SIAM Journal on Imaging Sciences, pp. 1993–2019, 2014.
5. Y. Shen, F. Lu, X. Cao, and H. Foroosh, “Video completion for perspective camera under constrained motion,” IEEE International Conference on Pattern Recognition, pp. 63-66, 2006.
6. M. Granados, K. Kim, J. Tompkin, J. Kautz, and C. Theobalt, “Background inpainting for videos with dynamic objects and a free-moving camera,” European Conference on Computer Vision (ECCV), pp. 682-695, 2012.
7. M. Roxas, T. Shiratori, and K. Ikeuchi, “Video completion via spatio-temporally consistent motion inpainting,” IPSJ Transactions on Computer Vision and Applications, pp. 98-102, 2014.
8. J.-B. Huang, S. B. Kang, N. Ahuja, and J. Kopf, “Temporally coherent completion of dynamic video,” ACM Transactions on Graphics (TOG), 2016.
9. Bokov A. A., Vatolin D.S. Metodika ob"ektivnoj ocenki kachestva vosstanovlenija fona v video (Methodology of objective quality assessment for video background reconstruction) // Cifrovaja obrabotka signalov (Digital signal processing), 2016, no. 3, pp. 26-33.
10. Y. Wexler, E. Shechtman, and M. Irani, “Space-time video completion,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2004.
11. T. Shiratori, Y. Matsushita, X. Tang, and S. B. Kang, “Video completion by motion field transfer,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp. 411–418, 2006.
12. C. Barnes, E. Shechtman, A. Finkelstein, and D. Goldman, “Patchmatch: A randomized correspondence algorithm for structural image editing,” ACM Transactions on Graphics (TOG), 2009.
13. K. Patwardhan and G. Sapiro, “Video inpainting under constrained camera motion,” IEEE Transactions on Image Processing, pp. 545–553, 2007.
14. S.-C.-S. Cheung, J. Zhao, and M. V. Venkatesh, “Efficient object-based video inpainting,” IEEE International Conference on Image Processing (ICIP), pp. 705-708, 2006.
15. M. Ebdelli, C. Guillemot, and O. Le Meur, “Examplar-based video inpainting with motion-compensated neighbor embedding,” IEEE International Conference on Image Processing (ICIP), pp. 1737-1740, 2012.
16. M. Ebdelli, O. Le Meur, and C. Guillemot, “Video inpainting with short-term windows: application to object removal and error concealment,” IEEE Transactions on Image Processing, pp. 3034–3047, 2015.
17. M. Liu, S. Chen, J. Liu, and X. Tang, “Video completion via motion guided spatial-temporal global optimization,” ACM International Conference on Multimedia, pp. 537-540, 2009.
18. N.-C. Tang, C.-T. Hsu, C.-W. Su, T.-K. Shih, and H.-Y.-M. Liao, “Video inpainting on digitized vintage films via maintaining spatiotemporal continuity,” IEEE Transactions on Multimedia, pp. 602-614, 2011.
19. Z. Xu, Q. Zhang, Z. Cao, and C. Xiao, “Video Background Completion Using Motion-Guided Pixel Assignment Optimization,” IEEE Transactions on Circuits and Systems for Video Technology, pp. 1393-1406, 2016.
20. M. Strobel, J. Diebold, and D. Cremers, “Flow and color inpainting for video completion,” German Conference on Pattern Recognition (GCPR), pp. 293-304, 2014.
21. Zachesov A.A., Erofeev M.V., Vatolin D.S. Ispol'zovanie kart glubiny pri vosstanovlenii fona v videoposledovatel'nostjah (Utilizing depth maps for background reconstruction in video) // Novye informacionnye tehnologii v avtomatizirovannyh sistemah: materialy nauchno-prakticheskogo seminara, 2015.
22. J. Herling and W. Broll, “High-quality real-time video inpainting with PixMix,” IEEE Transactions on Visualization and Computer Graphics, pp. 866–879, 2014.
23. S. Liu, L. Yuan, P. Tan, and J. Sun, “Steadyflow: Spatially smooth optical flow for video stabilization,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4209-4216, 2014.
24. P. Perez, M. Gangnet, and A. Blake, “Poisson image editing,” ACM Transactions on Graphics (TOG), pp. 313-318, 2003.
25. D. Min, S. Choi, J. Lu, B. Ham, K. Sohn, and M. N. Do, “Fast global image smoothing based on weighted least squares,” IEEE Transactions on Image Processing, pp. 5638-5653, 2014.
26. K. He and J. Sun, “Statistics of patch offsets for image completion,” European Conference on Computer Vision (ECCV), pp. 16-29, 2012.
27. A. Telea, “An image inpainting technique based on the fast marching method,” Journal of graphics tools, pp. 23–34, 2004.
28. J.-B. Huang, S. B. Kang, N. Ahuja, and J. Kopf, “Image completion using planar structure guidance,” ACM Transactions on Graphics (TOG), 2014.
29. The Foundry Nuke. https://www.thefoundry.co.uk/products/nuke/.
30. Pixel Farm PFClean. http://www.thepixelfarm.co.uk/pfclean/.
31. P. Weinzaepfel, J. Revaud, Z. Harchaoui, and C. Schmid, “Deepflow: Large displacement optical flow with deep matching,” IEEE International Conference on Computer Vision (ICCV), pp. 1385-1392, 2013.
The solution of overdetermined systems of linear equations in problems of image processing
Novikov A.I., e-mail: novikovanatoly@yandex.ru
The Ryazan State Radio Engineering University (RSREU), Russia, Ryazan
Keywords: : image, homographic matrix, point cloud, singular matrix decomposition, SVD method.
Abstract
The article deals with problems of digital image processing, in which there are redefined systems of linear algebraic equations (SLAE). Systems are redefined, the number of equations in which is much larger than the number of unknowns. One of these problems is related to the search for an optimal estimation of the homographic matrix,, which maps one image (the set of points) to the plane of the other. The level of the homographic matrix, is searched for a set of pairs of key points. The number of such pairs can reach several hundred, and the number of unknowns in the homology matrix is only 8 in the planar case and 15 when the clouds of points in space are combined. As a result, SLAE appears, where the number of equations is greater than the number of unknowns.
Over determined systems of equations can be inconsistent because of measurement errors, although often from the physical meaning of the problem it follows that a solution must exist. In this case, you are looking for a pseudo-solution of the system. A pseudo-solution can be sought by the method of least squares. It leads to normal SLAE. An alternative approach is to construct a singular decomposition of the main matrix of the system.
The simplest examples show the singularities of the solution of such systems by the method of least squares and the SVD method using the singular expansion of the matrix of the system of equations. the transition from SLAE with the basic matrix of full rank to the normal system of equations guarantees the finding of the solution of the system. The SVD method as compared to the method of least squares requires more computational operations. B I.S. ut it has a wider range of applications.
Examples of the solution of over determined systems of equations in real problems of image processing are given. Practical recommendations on finding pseudo-solutions of joint but uncertain systems of equations, as well as incompatible systems of equations in the classical sense, are offered.
References
1. A.I. Novikov, V.A. Sablina, M.B Nikiforov and A.A. Loginov. The Contour Analysis and Image-Superimposition Problem in Computer Vision Systems // Pattern Recognition and Image Analysis, 2015, Vol.25, No 1, pp. 73-80. - DOI: 10.1134/S1054661815020194.
2. A.I. Efimov, Algoritm poehtapnogo utochneniya proektivnogo preobrazovaniya dlya sovmeshcheniya izobrazhenij (Algorithm of step-by-step refinement of the projective transformation for combining images) / A.I.Efimov, A.I. Novikov // Computer Optics. – 2016. – Ò. 40, ¹ 1. – pp. 258-266. – DOI: 18287/2412-6179-2016-40-2-258-266.
3. Voevodin V.V., Kuznecov Ju.A.. Matricy i vychisleniya (Matrices and calculations). Ì.: Science, 1984. – 320 p.
4. Andrushevskij N.M. Analiz ustojchivosti reshenij sistem linejnyh algebraicheskih uravnenij (Analysis of the stability of solutions of systems of linear algebraic equations). Ì.: Izdatel'skij otdel fakul'teta VMiK MGU im. M.V. Lomonosova; MAKS Press, 2008. – 71 p.
5. Demmel J. Vychislitel'naya linejnaya algebra (Computational linear algebra). Theory and practice. Ì.: Mir, 1999. – 430 p.
6. Golub J.,Van Lown Ch. Matrichnye vychisleniya (Matrix Computing). Ì.: Mir, 1999. – 548 p.
7. Demidovich B.P., Maron I.A. Osnovy vychislitel'noj matematiki (Fundamentals of Computational Mathematics). Ì.: Science, 1966. – 664 p.
8. H.D. Ikramov. CHislennye metody dlya simmetrichnyh linejnyh sistem (Numerical methods for symmetric linear systems). Ì.: Science, 1988. – 160 p.
9. Il’in v, Poznyak Eh.G Linejnaya algebra (Linear algebra). Ì.: Science, 1984. – 294 p.
10. Holopov I.S. Algoritm korrekcii proektivnyh iskazhenij pri malovysotnoj semke (Algorithm for correcting projective distortions at low-altitude shooting)/ I.S. Holopov // Comp. Optical. -2017. –P. 41, ¹ 2. – pp. 284-290. – DOI: 10.18287/0134-2452-2017-41-2-284-290.
11. Besl P.J., McKey, Neil D. A Method for Registration of 3-D Shapes // IEEE Transactions on Pattern Analysis and Machine intelligence, Vol. 14, ¹ 2á 1992. pp. 239-256.
12. Du, S. Affine iterative closest point algorithm for point set registration/ S. Du, N. Zheng, S. Ying, J. Liu // Pattern Recognition Letters/ - 2010 – 31/ -P. 791-799.
13. A. Makovetskii, S. Voronin, V. Kober, D. Tihonkih. Affine registration of point-to-plane approach// Collection of Proceedings of the Third International Conference and Youth School ITNT – 2017, Samara: New equipment. 2017. pp. 684-688.
Improving the thematic processing of hyperspectral information
Tretiakov V.A.*, e-mail: vdtretiakov@gmail.com
Rizvanov A.A.*,
Krivoshein V.V.*, e-mail: krivosheinvv@tsniimash.ru
Krotkov A.Y.*, e-mail: krotkovay@tsniimash.ru
Danilov R.Y.**, Ph.D. in Biology, e-mail: daniloff.roman@yandex.ru
*Federal State Unitary Enterprise “Central Research Institute of Machine Building” (FSUE TSNIImash), Korolev, Russia
**State Scientific Institution “All Russian Research Institute of Biological Plant Protection”, Krasnodar, Russia
Keywords: hyperspectral measurements, reflectance, spectral features, statistical pattern recognition, error probability, training data.
Abstract
Thematic processing of hyperspectral information is based on the methods of statistical pattern recognition. This paper presents the process improvement of the two classes recognition of ground objects. For this purpose, assumptions are examined such as classification of the test sample, the equality of covariance matrices of two classes, and the independence of spectral features.
The methods for determining the minimum required number of the recognition process parameters, such as sample volume, number of the spectral informative features, as well as the method for determining the values of wavelengths corresponding to the spectral features, are developed under the given assumptions. The method of determining the minimal required number of the spectral informative features is based on using the given error probability of recognition of two ground objects classes. The method for determining the wavelengths corresponding to the spectral features is based on one-sample t-test. The method of determining the minimum required sample volume is based on two-sample Fisher test. To verify the developed methods, ground experimental hyperspectral measurements were carried out on the test site of Krasnodar.
In time to come, the sample of the minimum required number of spectra with the minimum required number of informative spectral features would move towards a more accurate sampling, which can be used as training data when recognizing classes of ground objects in hyperspectral images obtained from space.
Moreover, the methods presented in this paper will be used for algorithm development of statistical pattern recognition with minimum required recognition parameters values without given assumptions. This algorithm can be used to classify the ground objects with subtle spectral differences, such as the health wheat and wheat leaf rust.
References
1. Mader S., Vohland M., Jarmer T., Crop classification with hyperspectral data of the HyMAP sensor using different feature extraction techniques- Proceedings of the 2nd Workshop of the EARSeL SIG on Land Use and Land Cover. 28-30 September 2006.
2. Li C., Wang J., Wang L., Hu L., Gong P. Comparison of Classification Algorithms and training sample sizes in Urban Land Classification with Landsat Thematic Mapper Imagery –Remote Sensing. 2014. ¹6. P.964-983.
3. Doma M.L., Gomaa M.S., Amer R.A. Sensitivity of pixel-based classifiers to training sample size in case of high resolution satellite imagery – Journal of Geomatics. 2015. V.9. ¹ 1. P.53-58.
4. Thenkabail Prasad S., Lyon G. John, Huete A. Hyperspectral Remote Sensing of Vegetation, CRC Press, USA, 2011, 782 p.
5. Aktual'nye problemy sozdaniya kosmicheskikh sistem distantsionnogo zondirovaniya Zemli, Vtoraya mezhdunarodnaya nauchno-tekhnicheskaya konferentsiya, tez.dokl. (Actual problems of development of space systems for remote sensing of the Earth, the second international scientific technical conference, Abstracts of Papers), Moscow, «VNIIEM Corporation »JSC, 2014, pp. 118-125.
6. Barabash Yu.L., Varskii B.V., Zinov'ev V.T., Kirichenko V.S., Sapegin V.F. Voprosy statisticheskoi teorii raspoznavaniya, (Questions of statistical pattern recognition theory) Moscow, « Sovetskoe radio », 1967, 400 p.
7. Svein F., Deivis Sh. Distantsionnoe zondirovanie Zemli: kolichestvennyi podkhod (Remote sensing : a quantitative approach), Moscow «Nedra», 1983, 401 p.
8. Anderson T. Vvedenie v mnogomernyi statisticheskii analiz (An introduction to multivariate statistical analysis), Moscow, Gosudarstvennoe izdatel'stvo fiziko-matematicheskoi literatury, 1963, 500 p.
9. Akopov A.K., Baula G.G., Krivoshein V.V., Krotkov A.Yu., Tret'yakov V.A. Kosmonavtika i raketostroenie, 2015, ¹6 (85), pp. 45- 50.
The algorithm of integrating two images of different spectrums based on color synthesis
V.V. Shipko, Candidate of Technical Sciences, e-mail: shipko.v@bk.ru
Military scholastic-scientific center of the Air forces «Air forces academy named by prof. N.E. Zhukovskogo and Y.A. Gagarina», Voronezh, Russian Federation
Keywords: integrating images, multispectral images, color images, co-processing, color synthesis.
Abstract
Systems of multispectral recording have been progressively developing recently. Each reception channel of such systems has their own advantages and restrictions because of different principles on which they are formed and two-dimensional images are displayed.
The article presents the analysis of ways in which multispectral information is displayed and integrated. The private matter is integrating images of different spectrums that are gained in separated ranges, for example, visible and infra-red bands. The classic approach to integrating these images consists of getting a single half-tone image comprising distinctive features of original images. Moreover, all already developed algorithms do not allow as to identify spectrum-dependent objects qualitatively and the resulting integrated image is not always identified with high local contrast of those objects. This is due to the limitation of dynamic range of half-tone images and also due to the diversity of on-site-background situations.
The article offers original algorithms of integrating two digital half-tone images as a color image, this allow us to increase the local contrast of the resulting image containing elements of the original images from one scene, gained in different spectral ranges, and also to simplify identifying spectrum-dependent objects. From the examples, of integrating images using the algorithm it is clear that the color synthesis an effective method for complex decryption of images gained in different ranges of the electromagnetic spectrum. An important feature of the synthesis is that the result of the synthesis with its properties and all other things being equal, is not worse than any spectrozonal image which argues in favour of the use of it in order to reduce the decryption time with the loss of data excluded. The color synthesis is also advantageous in terms of computing costs, for it does not require the image being processed pixel by pixel, but it uses physiological characteristics of the human color vision.
The algorithm can be used in any multispectral servo systems in order to increase their informativeness.
References
1. Tarasov V.V., Yakushenkov Ju.G. Dvuh i mnogodiapazonnie optiko-electronnie sistemi s matrichnimi priemnikami izlucheniya (Dual band and multiband optoelectronic systems with matrix emission reciever). Ì.: Universitetskaya kniga; Logos. 2007. 192 p.
2. Deistvujushie i perspektivnie sputniki nabljudeniya zemli (Current and forward-looking Earth observation satellites) // Geomatika. 2013. no. 2. pp. 107?111.
3. Avtomatizirovannie sistemi nazemnih kompleksov sbora i obrabotki dannih vozdushnoj razvedki (Automated system of ground-based complex of collecting and processing information aerospace reconnaissance). Uchebnoe posobie / Pod redakciei. I.N. Beloglazova. Ì.: VVIA im. prof. N.E. Jukovskogo. 2003. 296 p.
4. Shipko V.V. O vozmojnih podhodah komplecsirovaniya ikonicheskoj informacii v mnogokanal’nih sistemah vozdushno-kosmicheskoj razvedki (Possible approaches of integrating iconic information in multichannel systems of aerospace reconnaissance). Aktual’nie voprosi issledovanij v avionike: teoriya, obsluzivanie, razrabotki (Topical issue of researches in avionics): sb. tezisov dokl. IV Vserossijskoj nauchno-prakticheskoj konferencii «AVIATOR», Voronezh, 16-17 fevralya 2017 ã., Voronezh: VUNC VVS «VVA», 2017. 232 p.
5. Patent RU 2451338 published 20.05.2012 ã., ÌÏÊ G06T 5/00.
6. Patent RU 2540778 published 10.02.2015 ã., ÌÏÊ G06T 5/50, G06K 9/46.
7. Sovremennie tehnologii obrabotki dannih distancionnogo zondirovaniya Zemli (Modern technologies of processing remote sensing of Earth data) / Pod redakciei V.V. Eremeeva. M.: Fizmatlit. 2015. 460 p.
8. Bogdanov A.P., Holopov I.S. Algoritmi formirovaniya cvetnogo kompleksirovannogo izobrajeniya iz mnogosperktralnih monohromnih na osnove metodov preobrazovaniya cvetov (Algorithms of creation a color integrated image from the multispectral monochromatic ones based on the color conversion method) / Cifrovaya obrabotka signalov (Digital signals processing). 2013. no. 3. pp. 26-32.
9. Bogdanov A.P., Pavlov O.V., Holopov I.S. Povishenie bistrodejstviya Algoritmov formirovaniya cvetnogo kompleksirovannogo izobrajeniya po signalam raznosperktralnih monohromnih datchikov (Increase of speed of algorithms of creating a color integrated image from the signals of multispectral monochromatic stnsors) / Cifrovaya obrabotka signalov (Digital signals processing). 2013. no. 3. pp. 33-37.
10. Gonsalis R., Vuds R. Cifrovaya obrabotka izobrajenij (Digital image processing). Ì.: Tehnosfera. 2005. 1072 p.
The analysis of the speed of search algorithms on images of the corresponding points using modern computer
A.E. Kuznetsov
A.S. Ryzhikov
The Ryazan State Radio Engineering University (RSREU), Russia, Ryazan, e-mail: foton@rsreu.ru
Keywords: points coregistration, correlation, SURF, graphics card.
Abstract
There are two approaches to finding similar points on a pair of pictures. The first, traditional, is based on the correlation comparison of the fragments of the analyzed image with the set of its images in the search zone of the other. The second approach involves comparing the descriptors of the local features of the analyzed and reference images. In this article, the speed of the algorithms examined on modern computer technology including a multi-core CPU (CPU) and a graphics card (GPU) is performed.The main stages of the work of the correlation-extreme algorithm and the algorithm based on SURF descriptors are considered. A theoretical and experimental evaluation of their performance is carried out using a multi-core CPU and video card.
To obtain experimental data on performance, multithreaded implementation of algorithms in C++ language using OpenCV primitives was performed. The performance of the implemented algorithms was tested on the Intel Xeon 2.4Ghz x 36 CPU configuration, NVIDEA Tesla k20c GPU (706 MHz x 2496).
The conclusion that an algorithm based on SURF descriptors can be effectively paralleled on modern computer technology. Its massive parallel processing implementation on both a multi-core CPU and with the use of video card resources makes it possible to create arrays of coordinates of the same points with a speed that is two orders of magnitude higher than the rate of correlation-extreme search, also realized in parallel. Another issue that determines the feasibility of applying the SURF algorithm is the reliability of the comparison. This requires additional research, which is planned to devote the next work.
References
1. Kuznecov A.E. Konvejernye tekhnologii potokovoj obrabotki dannyh ot vysokodetalnyh sistem DZZ// Tez. dokl. chetvertoj mezhdunarodnoj nauch.-tekhn. konf. «Aktualnye problemy sozdaniya kosmicheskih sistem distancionnogo zondirovaniya zemli». AO «Korporaciya «VNIIEHM», 2016. 220 p.
2. Kuznecov A.E., Poshekhonov V.I., Ryzhikov A.S. Tekhnologiya avtomaticheskogo kontrolya tochnosti geoprivyazki sputnikovyh izobrazhenij po opornym snimkam ot KA «Landsat-8» // Cifrovaja obrabotka signalov (Digital signal processing), 2015, no. 3, pp. 37–42.
3. Kuznecov A.E., Svetelkin P.N. Operativnoe sovmeshchenie spektrozonalnyh izobrazhenij pri cvetosinteze (Operating matching multispectral images while color synthesizing) // Vestnik Ryazanskogo gosudarstvennogo radiotekhnicheskogo universiteta (Vestnik of Ryazan State Radio Engineering University), 2007, no. 22, pp. 3-7.
4. Pratt K. Digital image processing // S. Wiley.1978. 750 p.
5. H. Bay, T. Tuytelaars, L. Van Gool. Surf: Speeded up robust features // Computer vision ECCV, 2006, pp 404-417.
Comparative assessment of processing methods of digital images gotten with the help of the thermal imager during unmanned aerial vehicles surveillance
Zaitsev A.V.,
Solovjov V.A.,
Kichulkin D.A.,
Krasavtsev O.O.
Smolensk Research and Innovation Center of Electronic Systems “Zavant” LLC
Smolensk, Russia
, e-mail: info@zavant.ru
Keywords: processing algorithm, unmanned aerial vehicle, infrared radiation, detection, digital image, convolution, spatial differentiation, spatial filtration, method combination.
Abstract
The article examines the matter of extraction of the small-sized unmanned aerial vehicle (UAV) image amid the background of cloudy sky radiation in infrared wavelength range (8-13 μm) from the video frames received from the output of thermal imager. For detection of that type of aerial targets there was a proposal to perform a pre-processing of the digital images with the intent of increasing the signal-to-noise ratio. Review of previous proposed methods of pre-processing was done and their positive and negative aspects were identified. In particular, these operations were reviewed: convolution of digital image with reference image of a small-sized aerial vehicle, spatial spectral filtration, every other frame subtraction, spatial differentiation, etc.
Based on the results from field studies, the effectiveness of each of them was assessed. Revealed that a method of every other frame subtraction had the utmost effectiveness in any kind of meteorological situation, however it required system’s optical axis to be held in stationary position. This causes difficulty when searching and tracking the aerial target. Conclusion was made of feasibility to use a combination of methods, which will provide effective operation of an optico-electronic UAV detector in any kind of meteorological situations.
Proposed a method of combining spatial differentiation distribution of brightness within the line of digital image with convolution operation with expected derivative distribution, determined by the image of a small-sized aerial vehicle. To evaluate the effectiveness of proposed method, the mathematical modeling and field studies were conducted of proposed method of processing of digital images, gotten with the help of thermal imager. This allowed identifying numerical rating of the probability of stable tracking of small-sized aerial target.
Analysis results showed that for a stable tracking of small-sized aerial target, a signal-to-noise ratio has to be not less than 3–4. Proposed method of processing allows achieving, when observing UAV in the background of cumulus clouds, a signs-to-noise ratio equal to 3.3. With other methods, this ratio does not exceed the value of 1.5. This gives the opportunity to conclude of a sufficient effectiveness of proposed combination of methods for processing digital images, gotten with the help of thermal imager when observing small-sized aerial objects.
References
1. Solovjov V.A., Leshchev M.S., Azarov V.S., Digital correction of infrared images with the purpose of improvement of surveillance of small-sized aerial vehicles // Materials from international scientific-practical conference “New science: state and ways of development”. Part 2. Sterlitamak, “Agency of international research”, 2016. 249 p.
2. Methods of image computer processing / With mod. V.A. Soifera. – M.: FISMATLIT, 2003. – 784 p.
3. Methods of automatic detection and tracking of objects. Image processing and control. / Alpatov B.A., Babayan P.V., Balashov O.E., Stepashkin A.I., – M.: Radiotechnika, 2008. – 176 p.
4. Alpatov B.A., Kataev A.A., Detection of moving object in the sequence of images with limitation of area and speed of a moving object // Digital processing of signals, no. 1. M.: Russian scientific-technical organization of radio, electronics and communication named after A.S. Popova, 2007. – P. 11–16.
5. Mitrofanov D.G., Soloviev V.A., Zaitsev A.V., Kichulkin D.A., Azarov V.S., Extraction of images of small-sized aerial objects on thermal imager frames with the used of convolution // Reports of fifth all-Russian conference “Electronic methods of obtaining, processing and visualizing information”. Ì.: Russian scientific-technical organization of radio, electronics and communication named after A.S. Popova, 2015. P. 176-179.
6. Soloviev V.A., Zaitsev A.V., Azarov V.S., Kichulkin D.A., Krasavtsev O.O., Research numerical characteristics of digital filter for suppression of sky radiation during surveillance of small-sized aerial targets in infrared range // Reports of 18th international conference “Digital processing of signals and its implementation”. Ì., Russian scientific-technical organization of radio, electronics and communication named after A.S. Popova, 2016. P. 682–687.
7. Mitrofanov D.G., Soloviev V.A., Zaitsev A.V., and other, Filtration of digital images received with the help of thermal imager, when detecting small-sized aerial objects. Reports of international conference “Radio electronic systems and systems for informational communication technologies”. Ì., Russian scientific-technical organization of radio, electronics and communication named after A.S. Popova, 2016. P. 301–306.
8. Gonsales R., Vuds R., Digital processing of images. M., Publisher “Technosfera”, 2006. – 1072 p.
9. Ganov V., Konushin A., Segmentation of moving objects in video stream. Electronic resource: http://cgm.computergraphics.ru/
10. Stauffer C., Grimson W. Learning patterns of activity using real time tracking. 2000.
Object distance estimation algorithm for real-time stereoscopic vision system
Korepanov S.E., e-mail: korepanov.s.e@rsreu.ru
Smirnov S.A., e-mail: smirnov.s.a@rsreu.ru
Strotov V.V., e-mail: strotov.v.v@rsreu.ru
The Ryazan State Radio Engineering University (RSREU), Russia, Ryazan
Keywords: stereoscopic vision system, object position estimation algorithm, template matching, performance indicator, disparity, distance.
Abstract
The article deals with compact stereoscopic vision systems for aerial applications (e.g. for small and middle sized UAVs). The architecture of the system of this kind has strong limitation on size, weight, power consumption. The processing unit for these systems is often based on FPGA. It provides excellent performance and low power consumption. The other side of this architecture is limitation in image processing algorithm implementation. For example, the recursive algorithms can’t be effectively implemented in FPGA.
The stack of the problem in compact aerial stereoscopic vision systems includes distance estimation task for non-cooperate object. In this work two distance estimation algorithms are proposed. First algorithm is based on 2D template image matching algorithm. It estimates the position of the object on the image, captured by the right camera using the object image that is extracted from the left camera image as template. The distance is calculated by object disparity. This algorithm is precise enough for most application, but its complexity increases with object size growth. Therefore real time work cannot be guaranteed.
The second algorithm is simplified version of the first algorithm. It based of 1D template matching. It uses the 1D cumulative images that produced by image summation by rows and columns. It causes the precision and reliability decreasing, but allows real time implementing in FPGA using pipelining.
The toggling procedure between these two algorithms is based on the object size and algorithm performance indicators analysis. If the object is larger than defined limit and the second algorithm is capable in current condition it is used for the distance estimation. The first algorithm is used otherwise. The performance indicator is based on the difference between the object and the background mean brightness.
The experimental examinations were processed using the database with a number of artificial video sequences with different background. On every sequence object changes its size from 7x7 up to image size (distance changes from 1000 to 100 meters). The proposed algorithm toggling procedure allows estimating distance in real time. The absolute distance estimate error in this case is less than 10 meters.
References
1. Tirri A.E., Fasano G., Accardo D. et al. Advanced Sensing Issues for UAS Collision Avoidance // International Conference on Application and Theory of Automation in Command and Control Systems, 12-19 (2012).
2. Lai J.S., Mejias L., Ford J.J. Airborne vision based collision detection system // Journal of Field Robotics ¹28(2), 137-157 (2010).
3. Richard K. Smith. Seventy-Five Years of Inflight Refueling // OhneVerlagsangabe. Washington. – 1998. – 86 p.
4. Li B., Mu C., Wu B. A survey of vision based autonomous aerial refueling for Unmanned Aerial Vehicles // Intelligent Control and Information Processing (ICICIP), 2012 Third International Conference on. – IEEE, 2012. – pp. 1-6.
5. Starovoitov E. Sovremennye tehnicheskie sredstva passivnoy opticheskoi lokacii // Sovremennaya electronica. – 2011. – ¹2. – P. 40-43.
6. Alpatov B.A., Babayan P.V., Balashov O.E., Stepashkin A.I. Metody avtomaticheskogo obnarujeniya i soprovojdeniya ob’ektov. Obrabotka izobrajeniy i upravlenie. M. : Radiotechnica, 2008. 176 pp.
7. Balashov O.E. Algoritm obnarujeniya nazemnyh prepyatstviy pri malovysotnyh poletah // Cifrovaya obrabotka signalov. – 2015. – ¹3. – S. 15-19.
8. Vizil'ter Yu.V., Jeltov S.Yu., Bondarenko A.V., Ososkov M.V., Morjin A.V. Obrabotka i analiz izobrajeniy v zadachah mashinnogo zreniya: Kurs lekciy i prakticheskih zanyatiy. – M.: Fizmatkniga, 2010. – 672 s.
9. Kartashev V.A., Efimov S.S. Issledovanie pogreshnostey izmereniya polojeniya ob"ekta stereosistemoy s maloy bazoy. Trudy konferencii «Tehnicheskoe zrenie v sistemah upravleniya -2012» // M.: Izd. IKI RAN, 2012. – s. 170-172.
10. Nussberger A., Grabner H., Gool L.V. Aerial Object Tracking from an Airborne Platform // International Conference on Unmanned Aircraft Systems (ICUAS), 1284-1293, 2014.
11. Kovacs L., Benedek C. Visual real-time detection, recognition and tracking of ground and airborne targets, Proceedings of SPIE-IS&T Electronic Imaging, Computational Imaging IX, vol. 7873, 1-12 (2011).
12. Balashov Î., Muraviev V., Strotov V. Aerial vehicles collision avoidance using monocular vision // Proc. SPIE 9988, Electro-Optical Remote Sensing X, 99880T
13. Alpatov B., Korepanov S., Strotov V. A composite algorithm for variable size object tracking for high performance FPGA-based on-board vision systems //SPIE Remote Sensing. – International Society for Optics and Photonics, 2014. – Ñ. 92470A-92470A-8
14. Baklickiy V.K., Bochkarev A.M. Metody fil'tracii signalov v korrelyacionno-ekstremal'nyh sistemah navigacii. – M.: Radio i svyaz', 1986. – 216 c.
15. Derenchenko L.P., Troickiy V.I. Povyshenie tochnosti korrelyacionno-ekstremal'nyh sistem metodom interpolyacii celevoy funkcii. // Voprosy radioelektroniki: Sb. nauch. tr. – M:, 1992. – s. 16-18.
16. Korepanov S.E., Smirnov S.A., Strotov V.V. Sravnenie treh kriteriev rabotosposobnosti mnogoetalonnogo algoritma ocenki koordinat ob’ektov // Cifrovaya obrabotka signalov . – 2016.¹3. – s.15-18
Image processing algorithms for road situation analysis
M.D. Ershov
N.Yu. Shubin
The Ryazan State Radio Engineering University (RSREU), Russia, Ryazan, e-mail: aitu@rsreu.ru
Keywords: video analysis, embedded systems, image processing, object detection, background subtraction, Radon transform, multi-agent systems.
Abstract
This paper describes road situation analysis problems for traffic flow management and ensuring safety. Efficient road traffic control requires traffic surveillance, estimation of traffic stream parameters, immediate detection of dangerous situations and their reporting to appropriate services.
The following image processing algorithms are proposed: road marking detection algorithm, vehicle detection and counting algorithm, stopped vehicle detection algorithm. Algorithms are designed to process images obtained from a stationary camera. In our work online processing is performed on the embedded platform of CCTV camera and provides reducing the amount of transmitted data and thus reducing the requirements for communication links and computing server.
The vehicle detection and counting algorithm requires preliminary definition of regions of interest for each road lane. In order to automate this process it is proposed to perform road marking detection for observed section of the road. For this purpose the source image is divided into small blocks which are processed using the integral vector Radon transform (IVRT). The lines found in each block are combined into long lines that are taken as road marking lines. The short lines are connected using the multi-agent approach.
The developed vehicle detection and counting algorithm requires the definition of special regions of interest (the sensors) in the image. Each sensor is divided into two zones (usually the entry zone and exit zone). This allows us to determine the movement direction of a passing car, as well as perform a rough estimation of it speed. Object detection is based on the background estimation in each zone. Also the algorithm takes into account the possibility of shadow condition and excessive lighting.
Stopped vehicle detection algorithm processes the entire image and is based on background subtraction technique. In the proposed algorithm an exponential filter is used to update the back-ground estimation. Our approach is based on current background estimation and a queue of back-ground estimations for various short time intervals. Then we subtract the earliest background estimation from the current background estimation for stopped object detection.
The developed algorithms were implemented and tested on the embedded platform of smart cameras. Analysis of presented results shows the efficiency of algorithms and developed software. We can conclude that presented algorithms work in real time under different observation condi-tions.
References
1. Al-Smadi M., Abdulrahim K., Salam R.A. Traffic Surveillance: A Review of Vision Based Vehicle Detection, Recognition and Tracking // International Journal of Applied Engineering Research. 2016. Vol. 11(1). P. 713-726.
2. Alpatov B., Babayan P., Ershov M., Strotov V. The implementation of contour-based object orientation estimation algorithm in FPGA-based on-board vision system // Proc. SPIE, High-Performance Computing in Geoscience and Remote Sensing VI. SPIE, 2016. Vol. 10007. 8 p.
3. Ershov M.D. Primenenie algoritmov obrabotki izobrazheniy dlya otsenki parametrov transportnyih potokov // Novyie informatsionnyie tehnologii v nauchnyih issledovaniyah. Ryazan, RSREU, 2016. P. 259-261.
4. Babayan P., Shubin N. Line detection in a noisy environment with weighted Radon transform // Proc. SPIE 9024, Image Processing: Machine Vision Applications VII, 902409. 2014. 6 p.
5. Vittikh V.A., Skobelev P.O. The multi-agent models of interaction in demand-resource networks // Automatica and Telemechanica.¬ 2003. P. 177-185.
6. Babayan P., Shubin N. Detection of curved lines and estimation of their parameters on images // 6th Mediterranean Conference on Embedded Computing (MECO), Bar. 2017. P. 1-4.
7. Alpatov B.A. Optimalnoe otsenivanie parametrov dvizhuschegosya ob'ekta v posledovatelnosti izobrazheniy // Avtometriya. 1994. No. 2. P. 32-37.
8. Alpatov B.A., Babayan P.V. Razrabotka algoritma slezheniya za fonovyim izobrazheniem dlya videokompyuternoy sistemyi obnaruzheniya i opredeleniya koordinat dvizhuschihsya ob'ektov // Problemyi matematicheskogo modelirovaniya i obrabotki informatsii v nauchnyih issledovaniyah: Sbornik nauchnyih trudov. Ryazan, RSREA, 2003. P. 3-15.
9. Benezeth Y., Emile B., Laurent H., Rosenberger C. Review and evaluation of commonly-implemented background subtraction algorithms // International Conference on Pattern Recognition. 2008. 4 p.
10. Piccardi M. Background subtraction techniques: a review // Proc. of IEEE International Conference on Systems, Man and Cybernetics. 2004. P. 3099-3104.
11. Heikkila J., Silven O. A real-time system for monitoring of cyclists and pedestrians // 2nd IEEE Workshop on Visual Surveillance. 1999. P. 74-81.
If you have any question please write: info@dspa.ru