|Digital Signal Processing||
Applied television system adaptation to the plots dynamic
The adaptation of the television system to the dynamics of the plot is based on the solution of the problem of minimizing errors in measuring the time-varying coordinates of objects with the speed limit of reading information from the photodetector matrices. Variable parameters are the image clarity and frame rate when they are discretely switched interchange, carried out on the basis of the principle of equality of dispersion of interelement and inter-frame increments of the video signal.
To eliminate the dependence of the control on the level of illumination of the scene, a transition from the traditional scheme with an estimate of the difference between the compared values to their ratio is made, which, due to the discreteness of the raster, changes when switching from state to state ("full clarity and low frame rate", "reduced clarity and high frame rate") four times (each of the estimated variances twice, but in opposite directions).
Ensuring the stability of the system of automatic control of the decomposition parameters of the measuring system requires the introduction of hysteresis when switching between the two States. The relative width of the hysteresis (threshold ratio) due to fluctuations in the observation statistics should be greater than the minimum possible value of 4. The optimal value of the relative hysteresis width is determined by the criterion of the maximum time spent within the interval [2, 1/2].
The stated concept of the interchange of clarity and frame rate is the realization of a new paradigm in the theory of applied television systems, which replaced the old paradigm of passive accounting for the reduction of resolution in the motion of the observed object. The optimization of the considered space television system is aimed at extracting information of maximum quality, taking into account the limitation of the bandwidth of the communication channels of the television camera and the on-board computer and/or the ground receiving terminal. The proposed development of the theory of synthesis of computer vision systems takes into account the influence of solid-state imaging technology on the methods of system analysis and synthesis, optimization, management, decision-making, information processing based on the principle of dominant information.
2. Legostaev V.P., Raushenbakh B.V. Avtomaticheskaya sborka v kosmose/ Kosmicheskie issledovaniya, 1969, no. 6. - pp. 803–813.
3. Mikrin E.A. Boortovye kompleksy upravleniya kosmicheskikh apparatov. M., MGTU im. N. EH. Baumana, 2014. – 245 p.
4. Bratslavets P. F., Rosselevich I. À., KHromov L. I. Kosmicheskoe televidenie. M.: Svyaz, 1973. – 248 p.
5. Legostaev V. P., Shmyglevskij I. P. Upravlenie sblizheniem kosmicheskikh apparatov na ehtape prichalivaniya. Upravlenie v kosmose. Ò. 2. Ì., Nauka, 1972. - pp. 218–228.
6. Tverdotel'naya revolyutsiya v televidenii: Televizionnye sistemy na osnove priborov s zaryadovoj svyaz'yu, sistem na kristalle i videosistem na kristalle/ Pod red. À. À. Umbitalieva i À. K. TSytsulina. - M.: Radio i svyaz', 2006. - 312 p.
7. Teoriya i praktika kosmicheskogo televideniya/ Umbitaliev À. À., Pyatkov V. V., Bobrovskij À. I. i dr. //Pod red. Umbitalieva À. À., TSytsulina À. K. SPb, NII televideniya, 2017. – 368 p.
8. Obnaruzhenie ob"ektov na zvyozdnom fone / Levko G. V., Bobrovskij À. I., Morozov À. V., TSytsulin À. K. // Voprosy radioehlektroniki, seriya Tekhnika televideniÿ, 2016, no. 2. - pp. 29–38.
9. Statisticheskij sintez upravleniya televizionnoj sistemoj, adaptivnoj k dinamike syuzheta / Umbitaliev À. À., Pyatkov V. V., Morozov À. V. i dr. // Voprosy radioehlektroniki, seriya Tekhnika televideniya, 2016, no. 1. - pp. 3–11.
10. Àdaptatsiya parametrov razlozheniya televizionnogo koordinatora tselej/Umbitaliev À. À., Pyatkov V. V., Bobrovskij À. I. i dr.// Voenno-nauchnaya konferentsiya «Àktual'nye nauchno-tekhnicheskie aspekty razrabotki, ispytanij i ehkspluatatsii sredstv raketno-kosmicheskoj oborony», SPb, VKÀ im. À. F. Mozhajskogo, 20.10.2017. pp. 183–188.
11. Veroyatnostnye kharakteristiki reshayushhej statistiki v televizionnoj sisteme, adaptivnoj k dinamike syuzheta/ Rogachyov V. À., Morozov À. V., Bobrovskij À. I. i dr. Voprosy radioehlektroniki, seriya Tekhnika televideniya, 2018, no. 1. – pp. 71–78
12. Khromov L. I., Tsytsulin À. K., Kulikov À. N. Videoinformatika. M., Radio i svyaz', 1991. – 192 p.
14. Rekursivnaya fil'tratsiya opornoj statistiki adaptivnoj televizionnoj sistemy/ Morozov À. V., Chepelev À. G., Bobrovskij À. I. i dr. // Trudy 14-oj Mezhdunar. konf. «Televidenie: peredacha i obrabotka izobrazhenij», SPb, 26–27 june 2018. SPb: Izd-vo «Tekhnolit», 2018. - p. 199-203.
15. Shamis À. L. Vektor ehvolyutsii. Zhizn', ehvolyutsiya. Myshlenie s tochki zreniya programmista. M.: Knizhnyj dom «LIBROKOM», 2013. – 200 p.
16. Optimizatsiya gisterezisa sistemy upravleniya telekameroj, adaptivnoj k dinamike syuzheta/ Bobrovskij À. I., Rogachyov V. À., Morozov À. V. I dr.// Trudy 14-oj Mezhdunar. konf. «Televidenie: peredacha i obrabotka izobrazhenij», SPb, 26-27 june 2018. SPb: Izd-vo «Tekhnolit», 2018. - pp. 40-44.
Ascending and descending pass SAR image fusion based on fuzzy logic
Keywords: SAR image, ascending pass, descending pass, orbit, fuzzy logic, fusion.
2. Chandrakanth R., Saibaba J., Varadan G., Ananth Raj P. Fusion of ascending and descending pass high resolution SAR data // Journal of Geomatics. 2014. Vol. 8(2). pp. 164–169.
Effective organization of mass coordinate transformations in the geometric processing of SAR images
2. Zlobin V.K., Eremeev V.V. Obrabotka ajerokosmicheskih izobrazhenij (Aerospace image processing). Moscow: FIZMATLIT, 2006. 288 p.
2. Kuang H., Chen J., Yang W., Zhu Y., Zhou J., Li Ch. Accurate compensation of stop-go approximation for high resolution spaceborne SAR using modified hyperbolic range equation // IEEE In-ternational Geoscience and Remote Sensing Symposium. 2014. pp. 462–465.
3. Prats-Iraola P., Scheiber R., Rodriguez-Cassola M., Wollstadt S., Mittermayer J., Brautigam B., Schwerdt M., Reigber A., Moreira A. High precision SAR focusing of TerraSAR X experimental staring spotlight data // IEEE International Geoscience and Remote Sensing Symposium. 2012. pp. 3576–3579.
4. Cumming I.G., Wong F.H. Digital processing of synthetic aperture radar data: algorithms and implementation. Artech House, 2005. 660 p.
It is reviewed in detail how the matrix of spectral signatures of image components was obtained using SQP method. Thanks to SQP the developed algorithm takes into account full limitations of positivity and additivity imposed on the coefficients of mixture decomposition.
We give some examples of algorithm verification in the treatment of experimental aerial hyperspectral images. As initial hyperspectral data, we took images received by the video-spectrometer made by Research and Production Association "Lepton". Etalon spectral signatures of objects, needed for the formation of feature values, were obtained as a result of ground-based measurements, using spectroradiometer FieldSpec®4 ASD. The experiments showed that the developed algorithm provides a reduction of false targets in 1.5-2 times in comparison with other linear spectral unmixing methods
2. S. Kraut, L.L. Scharf, L.T. McWhorter Adaptive Subspace Detectors // IEEE Trans. Signal Process. 49 (1). 2001. pp. 1-16.
3. Gladkikh B.A. Metody optimizatsii i issledovanie operatsii dlya bakalavrov informatiki. Ch. II. Nelineinoe i dinamicheskoe programmirovanie: uchebnoe posobie. (The study of optimization methods and operations for bachelors of computer science. Part II. Nonlinear and dynamic programming: tutorial). Tomsk: Izd-vo NTL. 2011. 264 p.
4. Klaus Schittkowski, Ya-xiang Yuan Sequential Quadratic Programming Methods. Wiley Encyclopedia of Operations Research and Management Science. 2011.
5. K.Schittkowski More test examples for nonlinear programming // Lecture notes in Economics and Mathematical Systems, Vol. 282, Springer. 1987.
6. Li J., Bioucas-Dias J. M. Minimum volume simplex analysis: a fast algorithm to unmix hyperspectral data // Proc. of IEEE International Conference on Geoscience and Remote Sensing Simposium (IGARSS). Boston, USA: IEEE. 2008. V. 3. pp. 250-253.
7. Chapurskii L.I., Markov A.V., Grigor'eva O.V., Saidov A.G., Astakhova E.I., Zhukov D.V., Baza dannykh KSYa v spektral'nom diapazone 400…1000 nm dlya osnovnykh tipov podstilayushchei poverkhnosti (pochv, rastitel'nosti, ledovykh i snezhnykh pokrovov, vodnoi poverkhnosti, ob"ektov iskusstvennogo proiskhozhdeniya), vstrechayushchikhsya v raionakh s"emki KA «Resurs-P» (Database reflectance the main types of underlying surface (soil, vegetation, ice and snow cover, water surface, objects of artificial origin) found in the areas of survey spacecraft "Resurs-P”. in the spectral range of 400 ... 1000 nm), Svidetel'stvo o gosudarstvennoi registratsii bazy dannykh, No. 2012621165, reg. 13.11.2012.
8. O.V. Grigor'eva, M.O. Ivanets, A.V. Markov, D.V. Zhukov Metody podgotovki formalizovannykh etalonnykh priznakov dlya identifikatsii ob"ektov po dannym giperspektral'noi kosmicheskoi s"emki (Preparation methods of formal etalon features for target detection using hyperspectral remote sensing data) // Materialy V Vserossiiskoi nauchno-tekhnicheskoi konferentsii «Aktual'nye problemy raketno-kosmicheskoi tekhniki» (V Kozlovskie chteniya). Samara: SamNTs RAN. 2017. T. 1. pp. 281-286.
9. Gonzalez, Rafael C, Woods, Richard E., Digital image processing. Pearson Education, Inc, 2008. 1072 p.
10. Kostrov B.V. Teoriya i metodologiya primeneniya sekventnogo analiza dlya obrabotki aerokosmicheskikh izobrazhenii (Theory and methodology for applying sequence analysis to aerospace image processing). Avtoreferat dissertatsii na soiskanie uchenoi stepeni doktora tekhnicheskikh nauk. Ryazan. 2012.
11. Zalmanzon L.A. Preobrazovaniya Fur'e, Uolsha, Khaara i ikh primenenie v upravlenii, svyazi i drugikh oblastyakh (Fourie, Walsh, Haar transforms and their application in control, communications and other fields). M.: Nauka. Gl.red.fiz.-mat.lit. 1989. 496 ð.
The aim of this work is to research application of CWT for motion compensation. For the given task, in the article the construction and properties of DWT and CWT which is based on the dual true complex wavelet transform (DTCWT) are considered. The principle of inter-frame coding on the example of wavelet-based video coder is shortly given. Motion compensation method for real videos using CWT is proposed. For evaluation of the proposed compensation method, this work also investigates reference methods, which are based on using the value of luma component (block matching (BM) and overlapped block matching (OBM), which is used in video codec DIRAC).
Working with high definition video "city" and "stockholm" (https://media.xiph.org/video/derf/), the following results were obtained:
In considered wavelet-based video codec Dirac, wavelet transform coefficients are divided into different frequency bands and simply quantized by uniform quantization. However, wavelet transform coefficients can be quantized considering the subjective sensitivity of the human eye to different spatial frequencies. Therefore, divided bands can be quantized by different quantization steps as accepted in the standard JPEG2000. Furthermore, there are possibilities of applying the nonlinear quantization algorithm Lloyd-Max and the entropy-constrained quantization method, which are mostly used for converting analog signals to digital signals. Due to the simplicity of quantization method implemented in this video codec, there exists a potential for finding a better quantization method.
The quantization methods mentioned above were adapted into considered wavelet-based video codec. Based on analyzing the processing results of various video types using different quantization methods, the optimized quantization method, which is the joint use of uniform quantization for some high-frequency bands and the algorithm Lloyd-Max for other bands, was proposed. The output data rate for given distortion levels is reduced noticeably by using the proposed method. The processing results for various types of video show that the proposed method provides up to 4.2% bitrate saving for low distortion level in the intra-frame coding mode, up to 9.8% bitrate reduction for low distortion level and up to 11.2% bitrate saving for medium (acceptable) distortion level in the inter-frame coding mode compared to the uniform quantization method implemented in this video codec.
2. William B. Pennebaker, Joan L. Mitchell, JPEG still image data compression standard (3rd ed.), Springer, 1993, - p. 291. ISBN 978-0-442-01272-4.
3. ITU-T Recommendation T.800 (2002-08) - Information technology – JPEG 2000 image coding system: Core coding system.
4. Dirac video codec // https://sourceforge.net/projects/Dirac/
5. S. Lloyd, Least squares quantization in PCM // IEEE Transactions on Information Theory, IT-28, 129–137, March 1982.
6. J. Max, Quantizing for minimum distortion // IRE Transactions on Information Theory, IT-6, 7–12, 1960.
7. P. A. Chou, T. Lookabaugh, and R. M. Gray. Entropy-constrained vector quantization // IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 31–42, Jan. 1989.
8. Jianguo Zhang Ling Shao Lei Zhang Graeme A. Jones. Intelligent video event analysis and understanding. Springer, 2010 edition. - 251 pages. ISBN 978-3-642-17554-1.
9. G.Y. Gryzov, A.V. Dvorkovich, Three-Channel Wavelet Transform for Video Compression Applications // 6th Mediterranean Conference on Embedded Computing MECO 2017, 11-15 June 2017, pp. 1-4.
10. V. P. Dvorkovich, A.V. Dvorkovich. Okonnye funktsii dlya garmonicheskogo analiza signalov (Window Function for the Harmonic Analysis of Signals). Moscow: Tekhnosfera, 2016, 216 p. ISBN: 978-5-94836-432-2.
The article provides an overview of several literary sources, in which it is shown that the assessment of visual image quality by individual measures is not objective. For this reason, it was concluded that the use of combined quality criteria, which operate with several particular indicators, is appropriate. It is shown that despite the obvious advantages of the well-known integral quality index (IQI), which operate with normalized values of brightness, standard deviation, number of brightness levels, contrast and entropy, its values for noisy images do not correlate with the results of the subjective perception of their quality. Introduced an integral-multiplicative quality criterion, operating with partial indicators: estimates of average brightness, standard deviation of brightness of high-frequency components and noise, as well as average values of local contrasts of the analyzed frame and its low-frequency component. The results of applying the study of the developed quality criterion to images of various spectral ranges (visible, short-wave and long-wave infrared) are analyzed. It is concluded that it is necessary to pre-compensate infrared image defects (dead pixels, structural noise) before calculating the IMQI. It is also shown that for images obtained as a result of nonlinear brightness transformations (for example, Multiscale Retinex), the quality index obtained by IMQI formula is overvalued.
The results of the semi-real experiment showed that, unlike the known IQI, the proposed IMQI for images decreases rather than increases with a high power of additive white gaussian noise.
Object detection and position estimation is based on multi-step image spatial filtering. This approach is expanded on multispectral imaging. The algorithm performance indicator based on object brightness is proposed.
The object recognition algorithm is based on outer contour descriptor matching. The proposed orientation estimation algorithm consists of 2 stages: learning and recognition. Learning stage is devoted to the exploring of studied objects. Using 3D model of the reference objects we can collect the set of training images by capturing 3D model from viewpoints evenly distributed on a sphere. The object contour can be produced using various border extraction techniques, active contour method etc.
The recognition stage is focusing on matching process between an observed image descriptor and the reference image descriptors. The contour descriptor of the object is shifted cyclically to archive the rotation invariance. The result of the matching produces the measure of the difference between the captured object and the n-th reference object from the database. The recognition stage includes the limited number of the operation and can be processed in the real time image processed systems. The source data for the recognition algorithm is a set of binary images that produced by object detection algorithm.
The results on the experimental examinations are given. The experimental examinations are performed using a set of natural multispectral video sequences. They show that detection true positive rate is better than 0,9 with false alarm rate is less 0,05. The recognition true positive rate exceeds 90%.
2. Lanir J. Maltz M., Rotman S.R. Comparing multispectral image fusion methods for a target detection task // Optical Engineering. – 2007. – Vol. 46(6). – P. 066402-1–066402-8
3. Hailiang Shi., Baohui Tian, Yuanzheng Wang Fusion of multispectral and panchromatic satellite images using Principal Component Analysis and Nonsubsampled Contourlet Transform / Seventh International Conference on Fuzzy Systems and Knowledge Discovery (FSKD). – 2010. – PP. 2312 – 2315.
4. Mitianoudis N., Stathaki T. Adaptive image fusion using ICA bases / Proceedings of the International Conference on Acoustics, Speech and Signal Processing. – Toulouse, 2006. – PP. II-829–II-832.
5. Kaarna A. Integer PCA and wavelet transforms for multispectral image compression / IEEE 2001 International Geoscience and Remote Sensing Symposium (IGARSS). – 2001. – Vol.4. – PP. 1853 – 1855.
6. A. Sarkar et al. A MRF model-based segmentation approach to classification for multispectral imagery // IEEE Transactions on Geoscience and Remote Sensing. – 2002. – Vol. 40, Issue 5. – pp. 1102-1113.
7. F. Samadzadegan. Data integration related to sensors, data and models // ISPRS Congress, Vol. XXXV, Proceedings of Commission IV, Istanbul, Turkey, 2004.– p. 569-574.
8. Vidya Manian, Luis O. Jimenez, Land cover and benthic habitat classification using texture features from hyperspectral and multispectral images // Journal of Electronic Imaging 16(2), 023011 (Apr–Jun 2007), pp. 1-12.
9. Babayan P.V., Smirnov S.A. Object tracking using the template matching algorithm for multispectral (visible and infrared band) visual systems // Digital signal processing – 2010.–Pp.18-21.
10. Alpatov B.A. Optimal moving object parameter estimation in image sequences. // Avtometriya. – ¹2. – Pp. 32-37.
11. Muraviev V.S., Muraviev S.I. Object extraction and position estimation algorithm for the cloudy backgrounds // The Bulletin of the RSREU. – 2007. – ¹21– Pp. 20-24.
12. Alpatov B.A., Babayan P.V., Smirnov S.A., Maslennikov E.A. The special orierntation prior estimation algorithm using the outer contour descriptor // Digital signal processing – 2014. -¹3. – Pp.43-46.
13. Repin V.G., Tartakovskiy G.P. Statistical synthesis in case of the apriority ambiguity and the information system adaptation – Moscow: Soviet Radio, 1977. – 432 pages.
14. Alpatov B.A., Babayan P.V., Balashov O.E. and Stepashkin A.I. The methods of the automatically object detection and tracking. Image processing and control – Moscow: Radiotechnika, 2008. – 176 pages.
15. Alpatov B.A., Babayan P.V., Smirnov S.A. The composite aerial object tracking algorithm. // The Bulletin of the RSREU. – 2011. – ¹37– Pp. 7-12.
16. Muraviev V. S., Smirnov S. A., Strotov V. V. Aerial vehicles detection and recognition for UAV vision system // Computer Optics. – 2017. – Vol. 41. – ¹. 4. – Pp. 545-551.
Modern artificial neural networks are able to detect and localize objects of known classes. This allows them to be used in various technical vision systems. The article contains a comparison of different neural network architectures that are used to solve the problem of object detection and recognition.
Neural network architectures for the detection and recognition of objects can be divided into two large groups. The first group includes architectures that process regions in the image (Region-based Convolution Neural Network – R-CNN). The second group includes architectures that process the entire image (You Only Look Once – YOLO; Single Shot MultiBox Detector – SSD).
In this work we compare three architectures (YOLO, Faster R-CNN, SSD) by the following criteria: processing speed, mean Average Precision (mAP), precision and recall. Five neural network detectors were trained for comparison purposes: YOLOv3; Faster R-CNN with the Inceptio-nResnet-2 network for feature extraction; Faster R-CNN with the Resnet-101 network for feature extraction; SSD with the MobileNet-1 network for feature extraction; SSD with the MobileNet-2 network for feature extraction.
During the experiment we used images containing objects of the classes “pedestrian” and “vehicle”. About 6,700 marked up images were used for training and 750 images for processing. The quality of object detectors was assessed by plotting the precision-recall curve, as well as graphs of precision, recall and F-measure for different threshold. Also to assess the quality depending on the training iteration we calculated the average precision (AP) metric for each class of objects and the mAP metric (average AP value over all classes). The area under precision-recall curve (AUC) and mAP was used as integral assessments of the detector accuracy. Computational efficiency was evaluated by processing images with a resolution of 720×468 on the personal computer with NVIDIA GeForce GTX 1070 graphics processor.
Faster R-CNN networks have demonstrated an advantage in accuracy. So, according to the experiment results, Faster R-CNN based on the InceptionResnet-2 network has the highest accuracy but the average processing time is much longer. The SSD architecture is the most suitable for real-time image processing (especially with MobileNet networks) but it must be borne in mind that high accuracy requirements usually cannot be satisfied. The detector based on neural network YOLOv3 has a mean accuracy and computational efficiency compared with other detectors.
2. Alpatov B.A., Babayan P.V. Image processing and recognition technologies in on-board technical vision systems // Vestnik of Ryazan State Radio Engineering University. – Ryazan. – 2017. – No. 2. – pp. 34-44 (in Russian).
3. Alpatov B.A., Babayan P.V., Balashov O.E., Stepashkin A.I. Methods for automatic detection and tracking of objects. Control and image processing. – Moscow: Radiotehnika, 2008. – 176 p. (in Russian).
4. Alpatov B.A., Babayan P.V., Ershov M.D. Vehicle Detection and Counting System for Real-Time Traffic Surveillance // Proceedings of 7th Mediterranean Conference on Embedded Computing (MECO). – IEEE, 2018. – pp. 120-123.
5. Gouk H.G.R., Blake A.M. Fast sliding window classification with convolutional neural networks // Proceedings of the 29th International Conference on Image and Vision Computing, New Zealand. – ACM, 2014. – pp. 114-118.
6. Boser B.E., Guyon I.M., Vapnik V.N. A training algorithm for optimal margin classifi-ers // Proceedings of the fifth annual workshop on Computational learning theory. – ACM, 1992. – pp. 144-152.
7. Redmon J., Divvala S., Girshick R., Farhadi A. You only look once: Unified, real-time object detection // Proceedings of the IEEE conference on computer vision and pattern recognition. – 2016. – pp. 779-788.
8. Redmon J., Farhadi A. YOLO9000: better, faster, stronger // arXiv preprint, ar-Xiv:1612.08242. – 2016. – 9 p.
9. Redmon J., Farhadi A. YOLOv3: An incremental improvement // Tech report, ar-Xiv:1804.02767. – 2018. – 6 p.
10. Bishop C.M. Pattern Recognition and Machine Learning. – Springer-Verlag, New York, 2006. – 738 p.
11. Ren S., He K., Girshick R., Sun J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks // Extended tech report, arXiv:1506.01497. – 2016. – 14 p.
12. Girshick R.B., Donahue J., Darrell T., Malik J. Rich feature hierarchies for accurate ob-ject detection and semantic segmentation // IEEE Conference on Computer Vision and Pattern Rec-ognition (CVPR). – 2014. – 21 p.
13. Girshick R. Fast R-CNN // IEEE International Conference on Computer Vision (ICCV). – 2015. – 9 p.
14. Uijlings J.R.R., van de Sande K.E.A., Gevers T., Smeulders A.W.M. Selective Search for Object Recognition // International Journal of Computer Vision. – 2013. – Vol. 104. – pp. 154-171.
15. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu Ch.-Y., Berg A.C. SSD: Sin-gle Shot MultiBox Detector // European Conference on Computer Vision (ECCV), Springer, Cham. – 2016. – Vol. 9905. – pp. 21-37.
16. Wan S., Chen Z., Zhang T., Zhang B., Wong K. Bootstrapping Face Detection with Hard Negative Examples // arXiv:1608.02236. – 2016. – 7 p.
17. Geiger A., Lenz P., Urtasun R. Are we ready for Autonomous Driving? The KITTI Vi-sion Benchmark Suite // Conference on Computer Vision and Pattern Recognition (CVPR). – 2012. – 8 p.
18. Cordts M., Omran M., Ramos S., Rehfeld T., Enzweiler M., Benenson R., Franke U., Roth S., Schiele B. The Cityscapes Dataset for Semantic Urban Scene Understanding // Conference on Computer Vision and Pattern Recognition (CVPR). – 2016. – 11 p.
The paper presents an algorithm of pathology detection and classification in endoscopic images. The algorithm is based on the use of the convolutional neural network SSD. This neural network allows you to detect and classify objects with the best indicators of the ratio of speed and quality of work among the currently existing approaches.
Training and testing of the developed algorithm was carried out on the NVIDIA DGX-1 supercomputer using endoscopic images from the test base, assembled together with the Yaroslavl Clinical Oncological Hospital.
In the framework of the study, the following was obtained: AP (average precision), mAP (mean average precision), precision-recall curves. The results of the research show that the proposed algorithm based on neural network SSD can be successfully used for the endoscopic image analyses in real medical practice, which is confirmed by the high level of similarity of the obtained results with the expert markup.
2. Bisschops R. et al Performance measures for upper gastrointestinal endoscopy: a European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Initiative // Endoscopy, 48(9), 2016, 843-64
3. Kuvayev R.O., Nikonov Ye.L., Kashin S.V., Kapranov V.A., Gvozdev A.A. Kontrol' kachestva endoskopicheskikh issledovaniy, perspektivy avtomatizirovannogo analiza endosko-picheskikh izobrazheniy (Quality control of endoscopic studies, prospects for automated analysis of endoscopic images) // Kremlevskaya meditsina. Klinicheskiy vestnik, 2, 2013, 51-56.
4. Lebedev A.A., Stepanova O.A., Yurchenko Ye.A., Khryashchev V.V. Razrabotka algoritmov analiza izobrazheniy dlya klassifikatsii patologiy slizistoy obolochki zheludka (Development of image analysis algorithms for the classification of pathologies of the gastric mucosa) // Tsifrovaya obra-botka signalov i yeye primeneniye (DSPA-2018): dokl. 20-y mezhdunar. konf. – Moskva, 2018. T. 2. S. 644-649.
5. Batukhtin D.M., Peganova Ye.V., Mitrakova N.N., Rozhentsov A.A., Furman YA.A. Analiz uzkospektral'nykh endoskopicheskikh izobrazheniy na vnutrenney poverkhnosti pishchevoda (Analysis of narrow-spectrum endoscopic images on the inner surface of the esophagus) // Vestnik Povolzhskogo gosudarstvennogo tekhnologicheskogo universiteta. Seriya: radiotekh-nicheskiye i infokommunikatsionnyye sistemy, ¹ 4 (23), 2014. – s. 45 - 57.
6. Peganova Ye. V., Batukhtin D. M., Mitrakova N. N. Avtomatizirovannaya sistema segmentatsii uzkospektral'nykh izobrazheniy dlya optimizatsii endoskopicheskoy diagnostiki pri patologii pishchevoda (Automated system for segmentation of narrow-spectrum images to optimize endoscopic diagnostics for esophageal pathology) // EiKG. 2014. ¹3 (103).
7. O. A. Dunayeva, D. B. Malkova, M. L. Myachin, KH. Edel'sbrunner, Segmentatsiya klinicheskikh endoskopicheskikh izobrazheniy, osnovannaya na klassifikatsii vektornykh topologicheskikh priznakov (Segmentation of clinical endoscopic images based on the classification of vector topological features) // Model. i analiz inform. sistem, 20:6 (2013), 162–173.
8. Kovalenko D.A., Gnatyuk V.C. Assotsiatsiya stsen v endoskopicheskikh video (Association of scenes in endoscopic videos) // GraphiCon 2017: Obrabotka i analiz biomeditsinskikh izobrazheniy, Perm', 2017. – s. 269-274.
9. T. Tamaki, S. Sonoyama, T. Hirakawa, B. Raytchev, K. Kaneda, T. Koide, Computer-Aided Colorectal Tumor Classification in NBI Endoscopy Using CNN Features // in The Korea-Japan joint workshop on Frontiers of Computer Vision (FCV2016), 2016, pp. 61–65.
10. E. Ribeiro, A. Uhl, G. Wimmer, M. Hafner, Exploring Deep Learning and Transfer Learning for Colonic Polyp Classification // Computational and Mathematical Methods in Medicine, Volume 2016, 16 p.
11. Y. Bar, L. Wolf, I. Diamant, H. Greenspan, Deep Learning with Non-Medical Training Used for Chest Pathology Identification // In: SPIE Medical Imaging. 2015. p. 94140V-V-7.
12. Kuvayev R.O., Kashin S.V., Nikonov Ye.L., Itoh T., Gotoda T., Gono K. Ranniy rak zheludka: metodiki skrininga, endoskopicheskoy diagnostiki i maloinvazivnogo lecheniya (Early gastric cancer: screening, endoscopic diagnosis and minimally invasive treatment methods) // Do-kazatel'naya gastroenterologiya, 3 (3), 2014, 44-51.
13. Kuvayev R.O., Kashin S.V. Sovremennoye endoskopicheskoye issledovaniye zheludka s ispol'zovaniyem metodik uzkospektral'noy i uvelichitel'noy endoskopii: tekhnika provedeniya i algoritmy diagnostiki // Dokazatel'naya gastroenterologiya, 2 (5), 2016, 3-13.
14. Kuvayev R.O., Nikonov Ye.L., Kashin S.V. Helicobacter pylori-assotsiirovannyy khronicheskiy gastrit: novyye tekhnologii endoskopicheskoy diagnostiki (Helicobacter pylori-associated chronic gastritis: new technologies for endoscopic diagnosis) // Dokazatel'naya gastro-enterologiya, 4 (1), 2015, 19-24.
15. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, and S. E. Reed. SSD: Single Shot Multibox Detector. CoRR, abs/1512.02325, 2015.
16. Fully convolutional reduced VGGNet [Web-site]. URL: https://gist.github.com/weiliu89/2ed6e13bfd5b57cf81d6.
17. Canny, J. A Computational Approach to Edge Detection // IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. Pami-8, no. 6, 1986. pp. 679-698.
18. ImageNet Image Database [Web-site]. URL: http://www.image-net.org/.
The variational nonlinear method of the optical flow computation is the most accurate, but at the same time the most computationally intensive. Its implementation on a system-on-chip is a trade-off between the design difficulty and high performance hardware implementation. The variational approach of the optical flow computation is based on the solving a system of nonlinear partial differential equations. In the article technique for their approximation by finite differences is investigated. Nonlinear equations discretization leads to a system of linear algebraic equations, which can be numerically solved by an iterative Gauss-Seidel method (SOR method - Successive-Over-Relaxation) with enhanced convergence.
The proposed methodology was implemented in a system-on-chip containing a processor system (ARM processor) and programmable logic (FPGA). Appropriate hardware architecture based on the tasks distribution between software (SW) and hardware (HW) parts of the system for the optical flow computation was justified. Verilog hardware description language was used for the most effective hardware implementation.
The solution proposed in article is capable for a dense non-linear optical flow real-time computation and can act as SoC hardware-accelerator of the optical flow computation in various kinds of image processing tasks.
2. Elesina S.I., Nikiforov M.B., Loginov À.À., Kostyashkin L.N. Monografiya pod. red.. L.N. Kostyashkina, M.B. Nikiforova. Sovmeshenie izobrageniy v correlyacionno-extrimalnih navigacionnih sistemah (Image complexing in correlation-extreme navigation systems). .M: Radiotechnika, 2015, p. 208.
3. B. K. P. Horn and B. G. Schunck. Determining optical flow, Artificial Intelligence, 17:185–203, 1981.
4. Abukhalikov A.A., Belyakov P.V., Nikiforov M.B., Poisk kluchevih tochek na izobragenii (Key points detection on the image). Megdunarodnaya nauchno-tehnicheskaya i nauchno-metodicheskaya konferencia «Sovremennie tehnologii v nauke I obrazovanii » STNO-2016, 2016, pp. 103-108.
5. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. High accuracy optical flow estimation based on a theory for warping. In Proc. European Conf., Computer Vision, volume 4, 2004, pp. 25–36.
6. Obrabotka izobragenii v aviacionnih sistemah tehnicheskogo zreniya (Image processing in aviation technical vision systems)/ Pod. red. L.N.Kostyashkina, M.B. Nikiforova. Ì.: FIZMATLIT, 2016, pp. 28-32
7. D. Ustukov, Y. Muratov, M. Nikiforov, V. Gurov. Implementing one of stereovision algorithms on FPGA. Mediterranean Conference on Embedded Computing, Jun 2016.
8. A. Bruhn and J. Weicker. Towards ultimate motion estimation: combing highest accuracy with real-time performance. In Proc. 10th IEEE Int.Conf., Computer Vision, 2005, pp. 749–755.
9. A. Bruhn, J. Weickert, and C. Schnorr. Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Int. J. Computer Vision, 2005, 61:211–231.
10. M. Kunz, A. Ostrowski, P. Zipf. An FPGA-optimized architecture of Horn and Schunck optical flow algorithm for real-time applications. Field Programmable Logic and Applications (FPL), 2014 24th International Conference.
11. J. L. Martin, A. Zuloaga, C. Cuadrado, J. Lazaro, and U. Bidarte. Hardware implementation of optical flow constraint equation using fpgas. Computer Vision and Image Understanding, 2005, pp 462–490.
12. Z. Chai, H. Zhou, Z. Wang and D. Wu Using C to implement high-efficient computation of dense optical flow on FPGA-accelerated heterogeneous platforms. IEEE 14 International Conference on Field-Programmable Technology (FPT), 2014.
13. Ortega, James M. Introduction to Parallel and Vector Solution of Linear Systems, 1988.
14.Xilinx.Zynq-7000SoC. https://www.xilinx.com/support/documentation/user_guides/ ug479_7Series_DSP48E1.pdf.
15. Xilinx. Zynq-7000 SoC. http://www.xilinx.com/products/silicon-devices/soc/zynq-7000/index.htm.
16. Xilinx. Vivado Design Suite. http://www.xilinx.com/products/design-tools/vivado.
17. Larkin E.V. Modelirovanie processa distancionnogo upravlenia robotom (Remote robot control process simulation). Izvestiya TulGU. Technicheckie nauki, 2016, Vip. 12. P. 4, pp. 202-214
If you have any question please write: firstname.lastname@example.org