|Digital Signal Processing||
Improving the performance of the algorithm for generating a panoramic video in vision systems with a distributed aperture
An article provides a comparative analyze of four methods for bilinear interpolation computation, differing in the number of additions and multiplications and the number of records of auxiliary variables in RAM, as well as numerical methods aimed at reducing the asymptotic computational complexity of the multi-band blending algorithm. These methods consist in optimizing the spatial 2D box filtering algorithm by a two-stage 1D Box filtering (by row and column) with brightness accumulator and replacing the division operation with an arithmetic right shift and applying the morphological dilation algorithm based on construction of Manhattan distances map for blending mask forming. The latter allows us to reduce the computational cost of implementing morphological dilatation with a k×k pixels structural element almost k2 times.
It is noted that considered optimization methods are effective when implementing calculations on a single-core processors and not applicable for parallel computations, because processing for each pixel of the image is not uniform.
The results of experiments performed on a prototype of a distributed panoramic system with five cameras and RoI size of 1024x768 pixels showed that when implementing computations on single core PC processor, the use of the considered optimization algorithms allows to increase the speed of RoI forming by 5.4 times.
2. Prudnikov N.V., Shlishevsky V.B. Panoramnye optico-electronnye ustrojstva krugovogo i sektornogo obzora (All-round looking optoelectronic surveillance systems) // Vestnik SGUGiT, 2016, vol. 33, no. 1, pp. 148-161.
3. Belskiy A., Zhosan N., Brondz D., Gorbachev K., Grebenschikov V., Kargaev A. Day/night synthetic vision system // Photonics, 2013, vol. 38, no. 2, pp. 80-86.
4. Bagdasarova O.V., Bagdasarov A.A. Panoramnye sistemy krugovogo obzora v ustrojstvah otobrazhenija vtorichnoj informatsii komplexov avioniki i avtobazirovanija (Panoramic systems of a circular review in devices for displaying secondary information of avionics and auto placement complexes) // Phazotron, 2012, no. 3, pp. 28-33.
5. Arshakyan A.A. Panoramnoe nabljudenie stseny s borta letatel’nogo apparata (Scene panoramic observation from flying machine board) // Izvestija TulGU. Tehnicheskie nauki, 2013, vol. 4, pp. 144-151.
6. Shirokov R.I., Alekhnovich V.I. Uvelichenie uglov polja zrenija sostavnoj WEB-kamery metodom skleivanija izobrazhenij (Compound webcam angles of field of vision extension by a method of images pasting together) // Contenant, 2014, no. 4, pp. 10-23.
7. Silvestrova O.V. Otsenka effektivnosti sposobov tsvetovoj korrektsii dlja panoramnyh otobrazhenij s malorazmernymi ob’ektami (Color correction methods efficiency evaluation for panoramic images with small-size objects) // Tsifrovaja obrabotka signalov, 2015, no. 3, pp. 25-28.
8. Brown M, Lowe D. Automatic Panoramic Image Stitching using Invariant Features // International Journal of Computer Vision. 2007. Vol. 74(1). P. 59-73.
9. Szeliski R. Image alignment and stitching: a tutorial // Foundations and trends in computer graphics and vision. 2006. Vol. 2(1). P. 1-104.
10. Perazzi F., Sorkine-Hornung A., Zimmer H., Kaufmann P., Wang O., Watson S., Gross M. Panoramic video from unstructured camera arrays // Comput. Graph. Forum. 2015. Vol. 34(2). P. 57-68.
11. Agarwala A., Zheng K.C., Pal C., Agrawala M., Cohen M., Curless B., Salesin D., Szeliski R. Panoramic video textures // ACM Transactions on Graphics. Vol. 24(3). P. 821-827.
12. Shum H.-Y., Szeliski R. Construction of panoramic mosaics with global and local alignment // International J. of Computer Vision. 2000. Vol. 36(2). P. 101-130.
13. Hartley R., Zisserman A. Multiple view geometry in computer vision: 2nd edition. Cambridge: Cambridge University Press, 2003. 656 ð.
14. Faugeras O., Lustman F. Motion and structure from motion in a piecewise planar environment // International J. of Pattern Recognition and Artificial Intelligence. 1988. Vol. 2(3). P. 485-508.
15. Malis E., Vargas M., Deeper understanding of the homography decomposition for vision-based control, Technical report, INRIA, 2007.
16. Efimov A.I., Novikov A.I. An algorithm for multistage projective transformation adjustment for image superimposition // Computer Optics. 2016. Vol. 40, No. 2. P. 258-265.
17. Strotov V.V. Sravnenie dvyh strukturnyh algoritmov otsenki parametrov geometricheskih preobrazovanij izobrazhenij (The comparation of two structural image geometric transformation parameters estimation algorithms) // Tsifrovaja obrabotka signalov, 2013, no. 3, pp. 21-25.
18. Lowe D.G Distinctive Image Features from Scale-Invariant Keypoints // International Journal of Computer Vision. 2004. Vol. 60(2). P. 91-110.
19. Fischler M. Bolles R. Random Sample Consensus: a Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography // Communications of the ACM. 1981. Vol. 24(6). P. 381-395.
20. Novikov A.I. Sablina V.A., Nikiforov M.B., Loginov A.A. The contour analysis and image-superimposition problem in computer vision systems // Pattern Recognition and Image Analysis. 2015. Vol. 25(1). P. 73-80.
21. Kudinov I.A., Pavlov O.V., Kholopov I.S., Khramov M.Yu. The algorithm for a video panorama construction and its software implementation using CUDA technology // CEUR Workshop Proceedings. Vol. 1902. 2017. P. 37-42.
22. Gruzman I.S., Kirichuk V.S., Kosykh V.P., Peretyagin G.I., Spektor A.A. Cifrovaja obrabotka izobrazhenij v informatsionnyh sistemah (Digital image processing in information systems). Novosibirsk: NSTU Publisher, 2002. 352 p.
23. Krasil’nikov N.N. Tsifrovaja obrabotka 2D- i 3D-izobrazhenij (Digital processing of 2D- and 3D-images). SPb.: BHV-Peterburg. 2011. 608 p.
24. Parker J.A. Kenyon R.V., Troxel D.E. Comparison of interpolating methods for image resampling // IEEE Trans. on Medical Imaging. 1983. – Vol. 2(1). P. 31-39.
25. Li J., Su J., Zeng X. A solution method for image distortion correction model based on bilinear interpolation // Computer Optics. 2019. Vol. 43(1). P. 99-104.
26. Timofeeva N.E., Geraskin A.S. Issledovanie vozmozhnosti uluchshenija algoritma bilinejnoj interpoljatsii dlja korrektirovki tsifrovyh izobrazhenij primeneniem teorii polej orientatsii (Research of improved bilinear interpolation algorithm for correcting a digital image using the theory of orientation fields) // Vestnik VGU. Serija: sistemnyj analiz I informatsionnye tehnologii, 2018. no. 1, pp. 119-125.
27. Zhu Z., Lu J., Wang M., Zhang S., Martin R.R., Liu H., Hu S.-M. A comparative study of algorithms for realtime panoramic video blending // IEEE Trans. on image processing. 2018. Vol. 27, No. 6. P. 2952-2965.
28. Kanaeva I.A., Bolotova Yu.A. Color and luminance corrections for panoramic image stitching // Computer Optics. 2018. Vol. 42, No. 5. P. 885-897.
29. Jahne B. Digital image processing: 6th ed, revised and extended. New York: Springer, 2005. 585 p.
30. Myratov Ye.R., Ustukov D.I. Algoritm vyravnivanija osveschennosti ob’ektov na izobrazhenijah s potochnym vypolneniem (Algorithm of objects illumination alignment on images with pipeline) // Tehnicheskoe zrenie v sistemah upravlenija – 2017: tezisy dokladov. Ìoscow: IKI RAN, 2017, pp. 29-31.
31. Lukin A. Tips & Tricks: Fast Image Filtering Algorithms // GraphiCon: Proc. of GraphiCon'2007Russia conference, Moscow, June 23-27, 2007.
32. Jarosz W. Fast Image Convolutions [Ýëåêòðîííûé ðåñóðñ]. Ðåæèì äîñòóïà: http://elynxsdk.free.fr/ext-docs/Blur/Fast_box_blur.pdf. Äàòà îáðàùåíèÿ: 17.05.2017.
33. Pratt W.K. Digital image processing: 4th ed. New Jersey: John Wiley & Sons, 2007. 807 p.
34. Burt P., Adelson E. A multiresolution spline with application to image mosaics // ACM Transactions on Graphics. 1983. Vol. 2(4). P. 217-236.
35. Vizilter Yu.V., Zheltov S.Yu., Bondarenko A.V., Ososkov M.V., Morzhin A.V. Obrabotka i analiz izobrazhenij v zadachah mashinnogo zrenija: kurs lektsij i prakticheskih zanjatij (Image processing and analysis in problems of machine vision: a course of lectures and practical exercises). Moscow: Phizmatkniga, 2010. 672 p.
36. Efficiently implementing dilate and erode image functions [Ýëåêòðîííûé ðåñóðñ]. Ðåæèì äîñòóïà: https://blog.ostermiller.org/dilate-and-erode (äàòà îáðàùåíèÿ: 17.10.2017).
37. Minkowski H. Geometrie der zahlen. Leipzig und Berlin: Druck und Verlag von B.G. Teubner, 1910. 256 p.
38. Krause E.F. Taxicab geometry: an adventure in non-euclidean geometry. New York: Dover Publications, 2012. 96 p.
39. Kudinov I.A., Kholopov I.S., Khramov M.Yu. Tehnologija formirovanija panoramnyh raznospektral’nyh videoizobrazhenij dlja obzornyh avaitsionnyh optico-electronnyh sistem (The technology of the panoramic multi-spectral video forming aviation vision systems) // Trudy MAI, 2019, vol. 104, pp. 1-20.
Increase the dynamic range of the video system logical addition of digital images
Keywords: dynamic range, multi-channel shooting, logical addition, logical filter.
Dynamic range is one of the important characteristics of the shooting system. It determines the ability to display the entire variety of brightness simultaneously observed objects in the field of view. A large range of changes in the brightness of objects in real scenes does not always fit into the limited technical capabilities of the means of observation. If the shooting results are important, it is necessary to look for a simple way to increase the dynamic range of the resulting image.
An option for solving this problem can be to obtain a set of images of the same scene with different, complementary shooting parameters and their summation in an arithmetic device. The article considers another, more effective for digital systems method of summation with the help of simple logic functions. It performs the summation procedure without intermediate memorization and transfer operations. The formation of the total image is carried out at the rate of receipt of information.
The sequence of operations of the proposed method is described. An example of its implementation is given.
2. Garanin S. G., Zykov L. I., Klimov A. N. et al. Day observation of stars of weak brightness (7 m-8 m) from flat terrain. // Optical journal, vol. 84, No. 12, 2017.
3. Soyuz 22 is exploring Earth. / Joint publication of the USSR Academy of Sciences and Acad. of Sciences of the German democratic Republic – Moscow; Berlin: Nauka, 1980.
4. Vilenchik L. S., Goncharenko B. G., Kurkov I. N. et al. Method of expanding the dynamic range of transmitted gradations of brightness and/or illumination in a television system. Patent RU 2199827 // Bulletin of inventions No. 6, 2003.
5. Kartsev M. A. Arithmetic of digital machines. - M: Nauka, 1969.
6. Pospelov D. A. Logical methods of analysis and synthesis of schemes. - M: Energy, 1968.
7. Khanjyan O. A. Linear filtration based on the theory of symmetric functions // Radio engineering and electronics, vol. 8, 1986.
8. Kottsov V. A. Method of obtaining images with increased dynamic range. Patent RU 2578799 // Bulletin of inventions No. 9, 2016.
9. Bergson A. Creative evolution. - M: TERRA - Book club, 2001..
Modification of the U-Net convolutional neural architecture in the
multi-channel satellite images segmentation problem
The original architecture of the convolutional neural network U-Net has been modified by two encoders were used for RGB and NIR channels with their combination on the central unit. The final neural network architecture has 47 convolutional layers, 47 ReLU activation functions, 47 batches normalization operations, 1 sigmoid activation function, 10 upsampling operations, 5 downsampling operations, 11 feature merging operations.
9784 images from the Spacenet database with 250×250-pixel size was used for training the neural network. To assess the quality of the segmentation algorithm, the following metrics were used: Sorensen similarity coefficient (dice) and Jacquard index (IoU).
Conducting pretraining based on Spacenet images allowed us to achieve a Sorensen coefficient of 0.783 and a Jacquard index of 0.649. The proposed modification algorithm for the convolutional neural network can find application in the field of urban development to track the construction of large objects.
2. Vizilter Ju.V., Zheltov S.Ju. Obrabotka i analiz izobrazhenij v zadachah mashinnogo zrenija. Kurs lekcij i prakticheskih zanjatij (Image processing and analysis in machine vision problems. Lecture and practical course) // M.: Fizmatkniga, 2010. 672 p.
3. Alpatov B.A., Ershov M.D., Fel'dman A.B. Algoritm obrabotki izobrazhenij dlja sistemy kombinirovannogo videnija letatel'nogo apparata (Image Processing Algorithm for Aircraft Combined Vision System) // Cifrovaja obrabotka signalov. 2015. no 3. pp. 8–14.
4. Goodfellow Y., Bengio Y., Courville A. Deep Learning // The MIT Press, 2016. 800 p.
5. Bartalev S.A., Hovratovich T.S. Analiz vozmozhnostej primenenija metodov segmentacii sputnikovyh izobrazhenij dlja vyjavlenija izmenenij v leash (Analysis of the possibilities of applying satellite image segmentation methods to detect changes in forests) // Sovremennye problemy di-stancionnogo zondirovanija Zemli iz kosmosa. 2011. T. 8, no 1. pp. 44–62.
6. Zhang Z., Liu Q., Wang, Y. (2018). Road extraction by deep residual U-Net // IEEE Geoscience and Remote Sensing Letters, 15(5), pp. 749–753.
7. Zhang L., Zhang L., Du B. Deep learning for remote sensing data: A technical tutorial on the state of the art // IEEE Geoscience and Remote Sensing Magazine 4.2. 2016. pp. 22–40.
8. Chen Y., Jiang H., Li C., Jia X., Ghamisi P. Deep Feature Extraction and Classification of Hyperspectral Images Based on Convolutional Neural Networks, in IEEE Transactions on Geoscience and Remote Sensing, vol. 54, no. 10, pp. 6232–6251, Oct. 2016.
9. Zhu X., Tuia D., Mou L., Xia G-S., Zhang L., Xu F., Fraundorfer F. Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources, in IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 4, pp. 8–36, Dec. 2017.
10. Iglovikov V., Mushinskiy S., Osin V. Satellite imagery feature detection using deep convolutional neural network: A kaggle competition. 2017, arXiv preprint arXiv:1706.06169.
11. Grigoreva O.V. Subpikselnaja identifikacija obektov mestnosti po giperspektralnym dannym na osnove posledovatel'nogo kvadratichnogo programmirovanija i metoda ana-liza formy spektralnyh component (Subpixel identification of terrain objects using hyperspectral data based on sequential quadratic programming and a method for analyzing the shape of spectral components) // Cifrovaja obrabotka signalov. 2018. no 3. pp. 26–31.
12. Nikolenko C., Kadurin A., Arhangel'skaja E. Glubokoe obuchenie. SPb (Deep learning).: Piter, 2019. 480 p.
13. Erohin D.J., Ershov M.D. Sovremennye svertochnye nejronnye seti dlja obnaruzhenija i raspoznavanija obektov (Modern convolutional neural networks for object detection and recognition) // Cifrovaja obrabotka signalov. 2018. no 3. pp. 64–69.
14. Seferbekov S., Iglovikov V., Buslaev A., Shvets A. Feature Pyramid Network for Multi-Class Land Segmentation. Web: https://arxiv.org/pdf/1806.03510.pdf.
15. Ivanov E., Tishhenko I., Vinogradov A. Segmentacija mul'tispektral'nyh snimkov s pri-meneniem svertochnyh nejronnyh setej (Segmentation of multispectral images using convolutional neural networks) // Sovremennye problemy distancionnogo zondiro-vanija Zemli iz kosmosa. 2019. T. 16, no 1. pp. 25–34.
16. Khryashchev V., Pavlov V., Priorov A., Ostrovskaya A. Deep Learning for Region Detection in High-Resolution Aerial Images // 2018 IEEE East-West Design & Test Symposium (EWDTS), Kazan, 2018. pp. 1–5.
17. Khryashchev V., Ivanovsky L., Pavlov V., Ostrovskaya A., Rubtsov A. Comparison of Different Convolutional Neural Network Architectures for Satellite Image Segmentation // Proceedings of the FRUCT’23, Bologna, Italy, 2018. pp. 172–179.
18. Ronneberger O., Fischer P., Brox T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Medical Image Computing and Computer-Assisted Intervention (MICCAI), Springer, LNCS, vol. 9351, 2015. pp. 234–241.
19. Solovev R., Telpuhov D., Kustov A. Avtomaticheskaja segmentacija sputnikovyh snimkov na baze modificirovannoj svertochnoj nejronnoj seti UNET (Automatic segmentation of satellite images based on the modified UNET convolutional neural network) // Inzhenernyj vestnik Dona. 2017. no. 4.47 p.
20. Ivanovsky L., Khryashchev V., Pavlov V., Ostrovskaya A. Building Detection on Aerial Images Using U-NET Neural Networks // Proceedings of the FRUCT’24, Moscow, Russia, 2019. pp. 116–122.
21. Khryashchev V.V., Priorov A.L., Pavlov V.A., Ivanovsky L.I. Segmentacija obektov na sputnikovyh izobrazhenijah s ispolzovaniem svertochnyh nejronnyh setej (Segmentation of objects in satellite images using convolutional neural networks) // Uspehi sovremennoj radiojelektroniki. 2019. T. 73, no. 6. pp. 28–34.
22. SpaceNet Database, Web: http://explore.digitalglobe.com/spacenet.
23. Gulli A., Pal S. Deep Learning with Keras, Packt Publishing, 2017. 320 p.
24. Kingma D.P., Ba J. Adam: A Method for Stochastic Optimization, Web: https://arxiv.org/abs/1412.6980.
The use of preliminary filtration is due to the ability to suppress the noise component of signal as well as to emphasize the edges. The following approaches to preliminary filtering are considered: global and local approaches to contrast enhancement, morphology-based image correction, bilateral and guided filtering. Pre-processing is an optional step and also can be applied in conjunction with well-known edge detection algorithms.
The developed contour detection algorithm is based on the gradient calculation and on the application of a fuzzy inference system. The fuzzy inference system is based on the Mamdani or Takagi-Sugeno model. The considered systems allow determining the degree of membership of a point to a contour or a homogeneous area. The algorithm consists of the following steps:
The problems of configuring fuzzy inference systems are considered including setting membership functions for inputs and output as well as fuzzy rules.
Software implementation and experimental studies were carried out in the Matlab development environment. The proposed approaches were compared with the well-known Sobel and Canny edge detectors. The studies consisted in processing the base of real images and reference contour images. The quality of the algorithm work was numerically estimated by calculating the precision, recall and F-measure. The contrast enhancement allows to increase the F-measure by 1.5-4.5%. The best results were achieved using bilateral and guided filters. The F-measure has been increased by 6-15% compared with the results without preliminary processing.
The approach based on fuzzy inference systems allows to obtain higher values of quality metrics in comparison with the Sobel operator (7-11% in the F-measure) and the Canny algorithm (8-21% in the F-measure). The type and size of gradient filters have a significant influence on the work of approaches based on both the Mamdani and Takagi-Sugeno systems.
2. Alpatov B.A., Ershov M.D., Feldman A.B. Image processing algorithm for combined vision system of aircraft. Digital Signal Processing, 2015. No.3. P. 8-14 (in Russian).
3. Roberts L.G. Machine Perception of 3-D Solids. MIT Press, 1965. 82 p.
4. Sobel I., Feldman G. A 3x3 Isotropic Gradient Operator for Image Processing // Stanford Artificial Intelligence Project. 1968.
5. Prewitt J.M.S. Object Enhancement and Extraction, Picture processing and Psychopictorics. New York: Academic Press, 1970. P. 75-149.
6. Titov I.O., Emelyanov G.M. Vydelenie konturov izobrazhenija dvizhushhegosja ob'ekta. Vestnik Novgorodskogo gosudarstvennogo universiteta, 2010. No.55. P. 27-31 (in Russian).
7. Kirsch R.A. Computer determination of the constituent structure of biological images // Computers and Biomedical Research. 1971. Vol.4. P. 315-328.
8. Robinson G.S. Edge detection by compass gradient mask // Computer Graphics and Image Processing. 1977. Vol.6. P. 492-501.
9. Marr D.C., Hildreth E.C. Theory of Edge Detection // Proceedings of the Royal Society of London. Series B, Biological Sciences. 1980. Vol.207. P. 187-217.
10. Canny J.F. A Computational Approach to Edge Detection // IEEE Transactions on Pattern Analysis and Machine Intelligence. 1986. Vol.8. P. 679-698.
11. Vizilter Y.V., Zheltov S.Y. Image Processing and Analysis in Computer Vision Tasks. Moscow, Fizmatkniga, 2010. 672 p. (in Russian).
12. Mitchell M. An introduction to genetic algorithm. MIT Press, 1996. 158 p.
13. Babayan P.V., Shubin N.Y. Line detection in a noisy environment with weighted Radon transform // Proc. SPIE 9024, Image Processing: Machine Vision Applications VII. 2014. Vol.902409. 6 p.
14. Mamdani E.H., Assilian S. An experiment in linguistic synthesis with a fuzzy logic controller // International Journal of Man-Machine Studies. 1975. Vol.7. No.1. P. 1-13.
15. Sugeno M. Industrial applications of fuzzy control. Elsevier Science Ltd, 1985. 278 p.
16. Paris S., Hasinoff S.W., Kautz J. Local Laplacian Filters: Edge-Aware Image Processing with a Laplacian Pyramid // Communications of the ACM, 2015. Vol. 58. No. 3. P. 81-91.
17. Szeliski R. Computer Vision: Algorithms and Applications. Springer, 2010. 979 p.
18. Zuiderveld K. Contrast Limited Adaptive Histogram Equalization // Graphic gems IV, 1994. P. 474-485.
19. Tomasi C., Manduchi R. Bilateral filtering for gray and color images // 6th International conference on computer vision. IEEE, 1998. P. 839-846.
20. He K., Sun J., Tang X. Guided Image Filtering // IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013. Vol. 35. P. 1397-1409.
21 .Arbelaez P., Maire M., Fowlkes C., Malik J. Contour Detection and Hierarchical Image Segmentation // IEEE TPAMI. 2011. Vol.33. No.5. P. 898-916.
It should be noted that modern technologies and the possibility of parallelization for calculations allow us to perform motion compensation with more complex but more accurate cost functions. In this paper, for the traditional motion compensation method we propose a modification which applies other cost functions.
This paper is dedicated to the research and development of prediction quality improvement of traditional motion compensation method for video codecs based on inter-frame block motion compensation. The paper proposes a new motion compensation method with the use of approximation functions with additional parameters. For applying the considered approximation functions in the video codec, the subtasks associated with the accuracy and the transmission method of additional parameters have been solved. Based on the results of high-definition video processing with the approximation functions studied, the best suitable function has been chosen. Application of the proposed method significantly reduces the amount of compressed data (from 15% to 34%) at a given quality of the reconstructed frame for high-definition videos.
2. x.264 video codec // http://www.videolan.org/developers/x264.html
3. x.265 video codec // http://x265.org/
4. Dirac video codec // https://sourceforge.net/projects/dirac/
5. ITU-T Recommendation H.265, High efficiency video coding, 04/2013 // ISO/IEC FDIS 23008-2, Information technology – High efficiency coding and media delivery in heterogeneous environments – Part 2: High efficiency video coding (MPEG-H HEVC).
6. Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, Thomas Wiegand, Overview of the High Efficiency Video Coding (HEVC) Standard // IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, #12, 12/2012, pp. 1649-1668.
7. Jill M. Boyce, Yan Ye, Jianle Chen, Adarsh K. Ramasubramonian. Overview of SHVC: Scalable Extensions of the High Efficiency Video Coding Standard. IEEE Trans. Circuits Syst. Video Techn. 26(1): 20-34 (2016).
8. Jianguo Zhang Ling Shao Lei Zhang Graeme A. Jones. Intelligent video event analysis and understanding. Springer, 2010 edition. - 251 pages. ISBN 978-3-642-17554-1.
9. V. A. Gritsenko, E. V. Belosevic, E. K. Artemeva. Mathematical methods in geography: Textbook / Kaliningrad University – Kaliningrad, 1999. – 75 p. – ISBN 5-88874-151-5.
10. Ferster E., rents B. Methods of correlation and regression analysis. A guide for economists. Translation from German and Preface by V. M. Ivanova, Moscow: "Finance and statistics", 1983 - 304 p.
11. Test videos // https://media.xiph.org/video/derf/
Implementation features of the modified motion compensation method
This work is devoted to the implementation of the modified method in a specific video codec Dirac and the description of the features of the modified method. In comparison to traditional method which is implemented in the considered video codec Dirac, the proposed method utilizes the other cost function based on approximation function which contains additional parameters. The subtasks associated with the accuracy and the transmission method of additional parameters have been solved in order to adapt the proposed method in considered wavelet-based video codec Dirac. Working with various types of video , the following conclusions were obtained:
-Application of the proposed method in the considered video codec Dirac can reduce the bitrate up to 33% with good reconstructed video quality PSNR = 40 dB and up to 40% with excellent quality PSNR = 42 dB.
-The more difficult test videos (the changes in the scene are significantly different from the parallel movement), the better the proposed method in comparison with the traditional one.
2. Dirac video codec // https://sourceforge.net/projects/dirac/
3. V. A. Gritsenko, E. V. Belosevic, E. K. Artemeva. Mathematical methods in geography: Textbook / Kaliningrad University – Kaliningrad, 1999. – 75 p. – ISBN 5-88874-151-5.
4. Ferster E., rents B. Methods of correlation and regression analysis. A guide for economists. Translation from German and Preface by V. M. Ivanova, Moscow: "Finance and statistics", 1983 - 304 p.
5. Test videos // https://media.xiph.org/video/derf/
In the present work is offered voice activity detector (VAD), which increases the probability of correct pauses detection in human speech. The developed method was compared with Likelihood-Ratio-Based VAD (LR-VAD) method and G.729B VAD. The test data was 108 seconds of speech mixed with vehicle noise of 5, 10, 15, 20 and 25 dB SNR. The active and inactive regions of the speech material were marked manually. The proportions of the inactive and active regions of the speech material were 0.46 and 0.54, respectively.
The offered VAD proved to be better than the G.729B VAD for almost all SNR values. The G.729B VAD was better only at 5 dB SNR. The LR-VAD method proved to be better than the other methods by the “speech detection error” parameter, but in terms of the parameter “pause detection error” it was significantly worse than the others VAD methods. The lowest pause detection error for almost all SNR values was demonstrated by the VAD method proposed in this article. In further work, it is intended to improve the offered VAD to reduce speech detection error.
2. Kondoz A.M. Digital Speech. Coding for Low Bit Rate Communication Systems. – John Wiley & Sons, Ltd. 2004. – 442 p.
3. ITU-T (1996) A silence compression scheme for G.729 optimised for terminals conforming to ITU-T V.70, ITU-T Rec. G.729 Annex B.
4. ITU-T (1996) Coding of speech at 8 kbit/s using conjugate-structure algebraiccode excited linear prediction (CS-ACELP), ITU-T Rec. G.729.
5. J. Sohn, N. S. Kim, and W. Sung (1999) ‘A statistical model-based voice activity detection’, in IEEE Signal Processing Letters, 6(1):1–3.
6. Y. Ephraim and D. Malah (1984) ‘Speech enhancement using a minimum mean square error short-time spectral amplitude estimator’, in IEEE Trans. on Acoust., Speech and Signal Processing, 32(6):1109–20.
7. Y. Ephraim and D. Malah (1985) ‘Speech enhancement using a minimum mean square error log-spectral amplitude estimator’, in IEEE Trans. on Acoust., Speech and Signal Processing, 33(2):443–5.
8. O. Capp'e (1994) ‘Elimination of musical noise phenomenon with the Ephraim and Malah noise suppression’, in IEEE Trans. Speech and Audio Processing, 2(2):345–9.
9. V.V. Vitjazev, V.I. Rozov, V.A. Volchenkov, The Russian Federation patent for an invention ¹ RU 2436173 C1, Int. Cl. G10L 15/00, G10L 11/02, Method of Detecting Pauses in Speech Signals and Device for Realising Said Method. Proprietor – Ryazan State Radio Engineering University. Application: 2010124342/08, Date of filing: 15.06.2010, Date of publication: 10.12.2011 Bull. 34.
2. Zuev V.E., Kabanov M.V. Optics of atmospheric aerosol. Leningrad: Gidrometeoizdat, 1987. – 255 p. – Ch. 2.
3. O’Neill E.L. Introduction to statistical optics. Palo Alto, London: Addison-Wesley Publishing Company, 1963.
4. Papoulis Athanasios. Systems and transforms with applications in optic. New York: McGraw-Hill book company, 1968. – Ch. 5.
5. Levshin W.L. Spatial filtering in optical direction-finding systems. Moscow: Sovetskoye Radio, 1971. – Ch. 1.
6. Naoyoshi Nameda. Fog modulation transfer function and signal lighting. Lighting Research and Technology. 1992, no 24(2): 103-106.
7. Smirnov W.A. Theory and method for solving of optic image transfer problems in diffuse environments. Voprosy radioelectroniki – Technika Televideniya, 1965, no 6: 109-124.
If you have any question please write: firstname.lastname@example.org