How to Write Video Plugins

How to Write Video Plugins ===   Expand the Power of Open Source Programmatic Video Manipulating Tools > [name=郭學聰 Hsueh-Tsung Kuo] [color=red] > [time=Sun, 04 Aug 2024] ###### CC BY-SA 4.0 ---  ## Who am I? ![fieliapm](https://www.gravatar.com/avatar/2aef78f04240a6ac9ccd473ba1cbd1e3?size=2048 =384x384) Someone (who?) said: a game programmer should be able to draw cute anime character(?) ----  * A ~~programmer~~ coding peasant from game company in Taiwan. * Backend (and temporary frontend) engineer. * Usually develop something related to my work in Python, Ruby, ECMAScript, Golang, C#. * Built CDN-aware game asset update system. * Business large passenger vehicle driver. :bus: * Ride bike to help traffic jam. :racing_motorcycle: * Care about chaotic traffic in Taiwan. * Draw cute anime character in spare time. ---  ## Outline ----  4. Programmatic Video Manipulating Tools * Showcase * Structure * How to serve video frames? * Execution ----  5. Comparison * AviSynth * History * Feature * AviSynthPlus * Feature * VapourSynth * Origin * Feature * Change ----  6. Make Plugins * Build Environment * AviSynth & AviSynthPlus * Script Usage * Plugins * Parallelism * VapourSynth * Script Usage * Plugins * Parallelism ----  7. Design * Convention * API Interface Changed * Parallelism & Async 8. Conclusion 9. Resource 10. Q&A ---  ## Programmatic Video Manipulating Tools Non-linear video editor controlled entirely by scripting ----  ## Programmatic Video Manipulating Tools * Write script to describe how to edit video * Build filter graph * Serve video frames on the fly ----  ### Showcase ----  ### Structure ```graphviz digraph structure { nodesep=0.3 node [fontname=Courier,shape=box] edge [color=purple] subgraph cluster_AviSynth{ label="AviSynth" AviSynthCore AVSEffectFilter AVSSourceFilter } subgraph cluster_VapourSynth{ label="VapourSynth" VapourSynthCore VSEffectFilter VSSourceFilter } AviSynthCore [shape=component] AVSEffectFilter [shape=component] AVSSourceFilter [shape=component] VapourSynthCore [shape=component] VSEffectFilter [shape=component] VSSourceFilter [shape=component] VfW [label="<f0>VfW|<f1>DirectShow",shape=record] VideoPlayer [label="<f0>VideoPlayer|<f1>VideoEditor",shape=record] CLI->{AviSynthCore VapourSynthCore} [color=red] VideoPlayer->{VfW FileSystem} {VfW FileSystem}->{AviSynthCore VapourSynthCore} AviSynthCore->AVSEffectFilter AVSEffectFilter->AVSSourceFilter VapourSynthCore->VSEffectFilter VSEffectFilter->VSSourceFilter {rank=same;VideoPlayer CLI} {rank=same;FileSystem VfW} } ``` ----  ### Execution * Loading * Register functions * Parse the script * Build the filter graph * Runtime * Frame serving ----  ### Execution ```flow= st=>start: Open avs/vpy e=>end: Quit regfunc=>operation: Register functions parse=>operation: Parse the script build=>operation: Build the filter graph serve_frame=>subroutine: Serve frames event=>condition: Event release=>operation: Release resource st->regfunc(right)->parse(right)->build->event event(no@Close)->release->e event(yes@Frame request)->serve_frame(left)->event ``` ----  #### How to serve video frames? * Open it as video file * Hook AVIFile API * AVIFileOpen() * Hook file system * AVFS: ~~http://www.turtlewar.org/avfs/~~ ----  #### How to serve video frames? * Get frames by accessing API directly * Pipe CLI * avs2yuv (AviSynth) * VSPipe (VapourSynth) * Dedicated media framework filter * DirectShow * https://github.com/CrendKing/avisynth_filter * FFmpeg (AviSynth & VapourSynth) * GStreamer ---  ## Comparison ----  ### AviSynth & AviSynthPlus http://avisynth.nl/index.php/Main_Page ----  ### AviSynth - History * Developed by Ben Rudiak-Gould, Edwin van Eggelen, Klaus Post, Richard Berg and Ian Brabham in May 2000 * Picked up and maintained by the open source community which is still active nowadays * 2.5.x * Old school * 2.6.x * Enhanced API * New plugins framework * add 10bit 16bit support ----  ### AviSynth - Feature * Single-threaded * Dedicated script language * Trim video & audio together * Filters work on raw video frame data * BGR interleaved(packed) * DWORD aligned lines, need pitch info * YUV planar ----  ### AviSynthPlus - Feature * Default single-threaded * Use Prefetch() will get frames from its child node using multiple threads * Dedicated script language * Trim video & audio together * Filters work on raw video frame data * BGR interleaved(packed) * DWORD aligned lines, need pitch info * YUV planar * Support 10bit 16bit ----  ### VapourSynth * https://github.com/vapoursynth/vapoursynth * https://www.vapoursynth.com/ ----  ### VapourSynth - Origin * Written by Fredrik Mellbin (myrsloik) between jobs * Inspired by AviSynth ----  ### VapourSynth - Feature * Always multi-threaded * Dispatch a thread for every frame context * Parallel between frames and nodes * Python script * Generalized colorspaces * Filters work on color channel planes * one plane = one RGB/YUV channel * Need stride info between lines in a plane * Per frame properties * Support for video with format change * High performance & efficient memory usage ----  ### VapourSynth - Change * Since R55 * API v3.x :arrow_right: API v4 * Script function breaking change * Add audio support * But video and audio cannot be trimmed together ```python= # Assume NTSC standard framerate and 48kHz as default def framesToSamples(frameNum, framerate=29.97, samplerate=48000): return math.floor((samplerate/framerate)*frameNum) video = video[71:217640] audio = audio[framesToSamples(71):framesToSamples(217640)] ``` ---  ## Make Plugins ----  ### Build Environment * Native * Visual Studio (Community Edition) * Porting * GCC * MinGW ----  ### AviSynth & AviSynthPlus ----  ### AviSynth - Script Usage ```clike= video = AVISource("somevideo.avi") audio = WAVSource("music.wav") AudioDub(video, audio) # `last` will be audio dubbed result video Trim(0, 12000) ++ Trim(20000, 32000) ++ Trim(44000, 0) LanczosResize(320, 240) # AviSynth will serve the video in `last` ``` ----  ### AviSynth - Plugins API v2.5 ```cpp= class IClip { friend class PClip; friend class AVSValue; int refcnt; void AddRef() { InterlockedIncrement((long *)&refcnt); } void Release() { InterlockedDecrement((long *)&refcnt); if (!refcnt) delete this; } public: IClip() : refcnt(0) {} virtual int __stdcall GetVersion() { return AVISYNTH_INTERFACE_VERSION; } virtual PVideoFrame __stdcall GetFrame(int n, IScriptEnvironment* env) = 0; virtual bool __stdcall GetParity(int n) = 0; // return field parity if field_based, else parity of first field in frame virtual void __stdcall GetAudio(void* buf, __int64 start, __int64 count, IScriptEnvironment* env) = 0; // start and count are in samples virtual void __stdcall SetCacheHints(int cachehints,int frame_range) = 0 ; // We do not pass cache requests upwards, only to the next filter. virtual const VideoInfo& __stdcall GetVideoInfo() = 0; virtual __stdcall ~IClip() {} }; typedef AVSValue (__cdecl *ApplyFunc)(AVSValue args, void* user_data, IScriptEnvironment* env); extern "C" __declspec(dllexport) const char* __stdcall AvisynthPluginInit2(IScriptEnvironment * env) ``` ----  ### AviSynth - Plugins see AVSPlugin.cpp ----  ### AviSynth - Parallelism * SetFilterMTMode() * Prefetch() will get frames from its child node using multiple threads ----  ### AviSynth - Parallelism ```clike= SetFilterMTMode("DEFAULT_MT_MODE", 2) # or SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE) SetFilterMTMode("FFVideoSource", 3) # or SetFilterMTMode("FFVideoSource", MT_SERIALIZED) FFVideoSource(...) Trim(...) QTGMC(...) ... # Enable MT! Prefetch(4) ``` ----  ### AviSynth - Parallelism ```graphviz digraph structure { nodesep=1.0 node [fontname=Courier,shape=record] edge [color=purple] QTGMC [label="<0>QTGMC|<1>#1|<2>#2|<3>#3"] Trim [label="<0>Trim|<1>#1|<2>#2|<3>#3"] FFVideoSource [label="<0>FFVideoSource|<1>#101|<2>#102|<3>#103"] AviSynthCore->Prefetch Prefetch->QTGMC:1 Prefetch->QTGMC:2 Prefetch->QTGMC:3 QTGMC:1->Trim:1 QTGMC:2->Trim:2 QTGMC:3->Trim:3 Trim:1->FFVideoSource:1 Trim:2->FFVideoSource:2 [color=pink,style=dashed] Trim:3->FFVideoSource:3 [color=pink,style=dashed] } ``` ----  ### AviSynth - Parallelism Since AviSynthPlus 3.6 ```clike= # Filtering A Prefetch(1,4) # Filtering B Prefetch(4) # Filtering C Prefetch(1,4) ``` ----  ### VapourSynth ----  ### VapourSynth - Script Usage Until R54 ```python= from vapoursynth import core video = core.ffms2.Source(source='Rule6.mkv') video = core.std.Transpose(video) video.set_output() ``` ----  ### VapourSynth - Script Usage Since R55 ```python= from vapoursynth import core video = core.bs.VideoSource(source='filename.mkv') audio = core.bs.AudioSource(source='filename.mkv') video = core.std.FlipHorizontal(video) audio = core.std.AudioGain(audio,gain=2.0) video.set_output(index=0) audio.set_output(index=1) ``` ----  ### VapourSynth - Plugins API v3.x ```cpp= typedef void (VS_CC *VSFilterInit)(VSMap *in, VSMap *out, void **instanceData, VSNode *node, VSCore *core, const VSAPI *vsapi); typedef const VSFrameRef *(VS_CC *VSFilterGetFrame)(int n, int activationReason, void **instanceData, void **frameData, VSFrameContext *frameCtx, VSCore *core, const VSAPI *vsapi); typedef void (VS_CC *VSFilterFree)(void *instanceData, VSCore *core, const VSAPI *vsapi); typedef void (VS_CC *VSPublicFunction)(const VSMap *in, VSMap *out, void *userData, VSCore *core, const VSAPI *vsapi); VS_EXTERNAL_API(void) VapourSynthPluginInit(VSConfigPlugin configFunc, VSRegisterFunction registerFunc, VSPlugin* plugin) ``` ----  ### VapourSynth - Plugins see VSPlugin.cpp ----  ### VapourSynth - Parallelism * Call source filter GetFrame() once * Call non-source filter GetFrame() twice * activationReason = arInitial * vsapi->requestFrameFilter() * To queue the frame context into global task list * activationReason = arAllFramesReady * vsapi->getFrameFilter() * To get result frame from the frame context ----  ### VapourSynth - Parallelism * Thread pool will dispatch every task (frame context) * Locate the filter from the frame context * Trigger request/get frame of the filter * Store the frame to the frame context ----  ### VapourSynth - Parallelism ```python= import vapoursynth as vs video = vs.core.video_input_source.VideoInputSource(0, 'USB', 1280, 720) bilateral_video = vs.core.bilateral.Bilateral(video, sigmaS=20.0, sigmaR=0.1) bilateral_video.set_output() ``` ----  ### VapourSynth - Parallelism ```graphviz digraph structure { nodesep=1.0 node [fontname=Courier,shape=component] edge [color=purple] Output [shape=house] FrameContextQueue [shape=cylinder] Output->FrameContextQueue [color=red,style=dashed] FrameContextQueue->Bilateral [color=red,style=dashed] Bilateral->FrameContextQueue [color=blue,style=dashed] FrameContextQueue->VideoInputSource [color=blue,style=dashed] VideoInputSource->FrameContextQueue [color=purple] FrameContextQueue->Bilateral [color=purple] Bilateral->FrameContextQueue [color=orange] FrameContextQueue->Output [color=orange] } ``` red :arrow_right: blue :arrow_right: purple :arrow_right: orange ----  ### VapourSynth - Parallelism Implementation vapoursynth/src/core/vsthreadpool.cpp ----  ### VapourSynth - Parallelism # Cache implicitly inserted as parent node of every filter without flag *nfNoCache* at creating vapoursynth/src/cython/vapoursynth.pyx ----  ### VapourSynth - Parallelism * Spatial filter * fmParallel: directly get a frame * Temporal filter * FIR * fmParallel: directly get frames * Cache bless you :rolling_on_the_floor_laughing: * IIR * fmParallelRequests or fmUnordered ----  ### VapourSynth - Parallelism ```graphviz digraph structure { nodesep=1.0 node [fontname=Courier,shape=record] edge [color=purple] FIR [label="<0>FIR|<2>#2"] IIR [label="<0>IIR|<1>#1|<2>#2|<3>#3"] Child1 [label="<0>Child1|<1>#1|<2>#2|<3>#3"] Child2 [label="<0>Child2|<1>#1|<2>#2|<3>#3"] FIR:2->Child1:1 FIR:2->Child1:2 FIR:2->Child1:3 IIR:3->IIR:2 IIR:3->IIR:1 IIR:3->Child2:1 IIR:3->Child2:2 IIR:3->Child2:3 } ``` ---  ## Design ----  ## Design Why? * AviSynth * IScriptEnvironment * VapourSynth * VSAPI * VSActivationReason ----  ### Convention * AviSynth * Register create() * create() return IClip * AviSynth C * Register create() * create() create a node with get_frame() * VapourSynth * Register create() * create() create a node with init(), getFrame(), free() ----  ### API Interface Changed * IScriptEnvironment * AviSynthPlus/avs_core/include/avisynth.h ```cpp= IScriptEnvironment* __stdcall CreateScriptEnvironment(int version = AVISYNTH_INTERFACE_VERSION); ``` ----  ### API Interface Changed * VSAPI * vapoursynth/include/VapourSynth.h ```cpp= VS_API(const VSAPI *) getVapourSynthAPI(int version) VS_NOEXCEPT; ``` ----  ### Parallelism & Async ```cpp= typedef const VSFrameRef *(VS_CC *VSFilterGetFrame)(int n, int activationReason, void **instanceData, void **frameData, VSFrameContext *frameCtx, VSCore *core, const VSAPI *vsapi); ``` * activationReason * arInitial * arAllFramesReady ----  ### Parallelism & Async async/await would be better? ```cpp= async const VSFrameRef* VS_CC VSFilterGetFrame(int n, int activationReason, void **instanceData, void **frameData, VSFrameContext *frameCtx, VSCore *core, const VSAPI *vsapi) { // ... const VSFrameRef *src = await vsapi->getFrameFilterAsync(n, d->node, frameCtx); // ... } ``` ----  ### Parallelism & Async * Based on: * fiber framework * thread framework * native support in language * ... ----  ### Parallelism & Async Ugly, but simple ```cpp= const VSFrameRef* VS_CC VSFilterGetFrame(int n, int activationReason, void **instanceData, void **frameData, VSFrameContext *frameCtx, VSCore *core, const VSAPI *vsapi) { if (activationReason == arInitial) { vsapi->requestFrameFilter(n, d->node, frameCtx); } else if (activationReason == arAllFramesReady) { const VSFrameRef *src = vsapi->getFrameFilter(n, d->node, frameCtx); // ... } } ``` ---  ## Conclusion ----  ### Suggestion * Think design reasons * Watch the details of parallel behavior ----  ### Slogan :hash: {發揮你的駭客精神吧|<big>Use your hacker spirit</big>} > [name=郭學聰 Hsueh-Tsung Kuo] [time=2024_08_04] [color=red] :notebook: ---  ## Resource ----  ### Reference * AviSynth wiki[^ref1] * VapourSynth Official[^ref2] * Doom9 forum[^ref3] [^ref1]:http://avisynth.nl/index.php/Main_Page [^ref2]:https://www.vapoursynth.com/ [^ref3]:https://forum.doom9.org/ ---  ## Q&A --- <style> .reveal { background: #FFDFEF; color: black; } .reveal h2, .reveal h3, .reveal h4, .reveal h5, .reveal h6 { color: black; } .reveal code { font-size: 18px !important; line-height: 1.2; } .progress div{ height:14px !important; background: hotpink !important; } // original template .rightpart{ float:right; width:50%; } .leftpart{ margin-right: 50% !important; height:50%; } .reveal section img { background:none; border:none; box-shadow:none; } p.blo { font-size: 50px !important; background:#B6BDBB; border:1px solid silver; display:inline-block; padding:0.5em 0.75em; border-radius: 10px; box-shadow: 5px 5px 5px #666; } p.blo1 { background: #c7c2bb; } p.blo2 { background: #b8c0c8; } p.blo3 { background: #c7cedd; } p.bloT { font-size: 60px !important; background:#B6BDD3; border:1px solid silver; display:inline-block; padding:0.5em 0.75em; border-radius: 8px; box-shadow: 1px 2px 5px #333; } p.bloA { background: #B6BDE3; } p.bloB { background: #E3BDB3; } /*.slide-number{ margin-bottom:10px !important; width:100%; text-align:center; font-size:25px !important; background-color:transparent !important; }*/ iframe.myclass{ width:100px; height:100px; bottom:0; left:0; position:fixed; border:none; z-index:99999; } h1.raw { color: #fff; background-image: linear-gradient(90deg,#f35626,#feab3a); -webkit-background-clip: text; -webkit-text-fill-color: transparent; animation: hue 5s infinite linear; } @keyframes hue { from { filter: hue-rotate(0deg); } to { filter: hue-rotate(360deg); } } .progress{ height:14px !important; } .progress span{ height:14px !important; background: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAAMCAIAAAAs6UAAAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAAyJpVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAAADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNiAoV2luZG93cykiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6QUNCQzIyREQ0QjdEMTFFMzlEMDM4Qzc3MEY0NzdGMDgiIHhtcE1NOkRvY3VtZW50SUQ9InhtcC5kaWQ6QUNCQzIyREU0QjdEMTFFMzlEMDM4Qzc3MEY0NzdGMDgiPiA8eG1wTU06RGVyaXZlZEZyb20gc3RSZWY6aW5zdGFuY2VJRD0ieG1wLmlpZDpBQ0JDMjJEQjRCN0QxMUUzOUQwMzhDNzcwRjQ3N0YwOCIgc3RSZWY6ZG9jdW1lbnRJRD0ieG1wLmRpZDpBQ0JDMjJEQzRCN0QxMUUzOUQwMzhDNzcwRjQ3N0YwOCIvPiA8L3JkZjpEZXNjcmlwdGlvbj4gPC9yZGY6UkRGPiA8L3g6eG1wbWV0YT4gPD94cGFja2V0IGVuZD0iciI/PovDFgYAAAAmSURBVHjaYvjPwMAAxjMZmBhA9H8INv4P4TPM/A+m04zBNECAAQBCWQv9SUQpVgAAAABJRU5ErkJggg==") repeat-x !important; } .progress span:after, .progress span.nyancat{ content: ""; background: url('data:image/gif;base64,R0lGODlhIgAVAKIHAL3/9/+Zmf8zmf/MmZmZmf+Z/wAAAAAAACH/C05FVFNDQVBFMi4wAwEAAAAh/wtYTVAgRGF0YVhNUDw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMy1jMDExIDY2LjE0NTY2MSwgMjAxMi8wMi8wNi0xNDo1NjoyNyAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOnN0UmVmPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VSZWYjIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC8iIHhtcE1NOk9yaWdpbmFsRG9jdW1lbnRJRD0ieG1wLmRpZDpDMkJBNjY5RTU1NEJFMzExOUM4QUM2MDAwNDQzRERBQyIgeG1wTU06RG9jdW1lbnRJRD0ieG1wLmRpZDpCREIzOEIzMzRCN0IxMUUzODhEQjgwOTYzMTgyNTE0QiIgeG1wTU06SW5zdGFuY2VJRD0ieG1wLmlpZDpCREIzOEIzMjRCN0IxMUUzODhEQjgwOTYzMTgyNTE0QiIgeG1wOkNyZWF0b3JUb29sPSJBZG9iZSBQaG90b3Nob3AgQ1M2IChXaW5kb3dzKSI+IDx4bXBNTTpEZXJpdmVkRnJvbSBzdFJlZjppbnN0YW5jZUlEPSJ4bXAuaWlkOkM1QkE2NjlFNTU0QkUzMTE5QzhBQzYwMDA0NDNEREFDIiBzdFJlZjpkb2N1bWVudElEPSJ4bXAuZGlkOkMyQkE2NjlFNTU0QkUzMTE5QzhBQzYwMDA0NDNEREFDIi8+IDwvcmRmOkRlc2NyaXB0aW9uPiA8L3JkZjpSREY+IDwveDp4bXBtZXRhPiA8P3hwYWNrZXQgZW5kPSJyIj8+Af/+/fz7+vn49/b19PPy8fDv7u3s6+rp6Ofm5eTj4uHg397d3Nva2djX1tXU09LR0M/OzczLysnIx8bFxMPCwcC/vr28u7q5uLe2tbSzsrGwr66trKuqqainpqWko6KhoJ+enZybmpmYl5aVlJOSkZCPjo2Mi4qJiIeGhYSDgoGAf359fHt6eXh3dnV0c3JxcG9ubWxramloZ2ZlZGNiYWBfXl1cW1pZWFdWVVRTUlFQT05NTEtKSUhHRkVEQ0JBQD8+PTw7Ojk4NzY1NDMyMTAvLi0sKyopKCcmJSQjIiEgHx4dHBsaGRgXFhUUExIREA8ODQwLCgkIBwYFBAMCAQAAIfkECQcABwAsAAAAACIAFQAAA6J4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93bqCA7f7TFaYoIFAMMwczB5EkTzJllEUttmIGoG5bfPBjDawD7CsJC67uWcv2CRov929C/q2ZpcBbYBmLGk6W1BRY4MUDnMvJEsBAXdlknk2fCeRk2iJliAijpBlEmigjR0plKSgpKWvEUheF4tUZqZID1RHjEe8PsDBBwkAIfkECQcABwAsAAAAACIAFQAAA6B4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93TqS40XiKSYgTLBgIBAMqE/zmQSaZEzns+jQ9pC/5dQJ0VIv5KMVWxqb36opxHrNvu9ptPfGbmsBbgSAeRdydCdjXWRPchQPh1hNAQF4TpM9NnwukpRyi5chGjqJEoSOIh0plaYsZBKvsCuNjY5ptElgDyFIuj6+vwcJACH5BAkHAAcALAAAAAAiABUAAAOfeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GKifWaddvNQAtszXYCxgR/Zy5jYTFeXmSDiIZGdQEBd06QSBQ5e4cEkE9nnZQaG2J4F4MSLx8rkqUSZBeurhlTUqsLsi60DpZxSWBJugcJACH5BAkHAAcALAAAAAAiABUAAAOgeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GuifWaddvNwMkZtmY7AWMEgGcKY2ExXl5khFMVc0Z1AQF3TpJShDl8iASST2efloV5JTyJFpgOch8dgW9KZxexshGNLqgLtbW0SXFwvaJfCQAh+QQJBwAHACwAAAAAIgAVAAADoXi63P7wmUmrnVGOzbvfRsYYXGGe6MmF4kEOaSGYMwq2LizHfDGwIlLPNKGZfi6gZmggEAy2iVPZEKZqzakq+1xUFFYe90lxTsHmim6HGpvf3eR7skYJ3PC5tyystc0AboFnVXQ9XFJTZIQOYUYFTQEBeWaSVF4bbCeRk1meBJYSL3WbaReMIxQfHXh6jaYXsbEQni6oaF21ERR7l0ksvA0JACH5BAkHAAcALAAAAAAiABUAAAOeeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMfFlA4hTITEMxkIBMOuADwmhzqeM6mashTCXKw2TVKQyKuTRSx2wegnNkyJ1ozpOFiMLqcEU8BZHx6NYW8nVlZefQ1tZgQBAXJIi1eHUTRwi0lhl48QL0sogxaGDhMlUo2gh14fHhcVmnOrrxNqrU9joX21Q0IUElm7DQkAIfkECQcABwAsAAAAACIAFQAAA6J4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93bqCA7f7TFaYoIFAMMwczB5EkTzJllEUttmIGoG5bfPBjDawD7CsJC67uWcv2CRov929C/q2ZpcBbYBmLGk6W1BRY4MUDnMvJEsBAXdlknk2fCeRk2iJliAijpBlEmigjR0plKSgpKWvEUheF4tUZqZID1RHjEe8PsDBBwkAIfkECQcABwAsAAAAACIAFQAAA6B4umv+MDpG6zEj682zsRaWFWRpltoHMuJZCCRseis7xG5eDGp93TqS40XiKSYgTLBgIBAMqE/zmQSaZEzns+jQ9pC/5dQJ0VIv5KMVWxqb36opxHrNvu9ptPfGbmsBbgSAeRdydCdjXWRPchQPh1hNAQF4TpM9NnwukpRyi5chGjqJEoSOIh0plaYsZBKvsCuNjY5ptElgDyFIuj6+vwcJACH5BAkHAAcALAAAAAAiABUAAAOfeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GKifWaddvNQAtszXYCxgR/Zy5jYTFeXmSDiIZGdQEBd06QSBQ5e4cEkE9nnZQaG2J4F4MSLx8rkqUSZBeurhlTUqsLsi60DpZxSWBJugcJACH5BAkHAAcALAAAAAAiABUAAAOgeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMd8MbAiUu802flYGIhwaCAQDKpQ86nUoWqF6dP00wIby572SXE6vyMrlmhuu9GuifWaddvNwMkZtmY7AWMEgGcKY2ExXl5khFMVc0Z1AQF3TpJShDl8iASST2efloV5JTyJFpgOch8dgW9KZxexshGNLqgLtbW0SXFwvaJfCQAh+QQJBwAHACwAAAAAIgAVAAADoXi63P7wmUmrnVGOzbvfRsYYXGGe6MmF4kEOaSGYMwq2LizHfDGwIlLPNKGZfi6gZmggEAy2iVPZEKZqzakq+1xUFFYe90lxTsHmim6HGpvf3eR7skYJ3PC5tyystc0AboFnVXQ9XFJTZIQOYUYFTQEBeWaSVF4bbCeRk1meBJYSL3WbaReMIxQfHXh6jaYXsbEQni6oaF21ERR7l0ksvA0JACH5BAkHAAcALAAAAAAiABUAAAOeeLrc/vCZSaudUY7Nu99GxhhcYZ7oyYXiQQ5pIZgzCrYuLMfFlA4hTITEMxkIBMOuADwmhzqeM6mashTCXKw2TVKQyKuTRSx2wegnNkyJ1ozpOFiMLqcEU8BZHx6NYW8nVlZefQ1tZgQBAXJIi1eHUTRwi0lhl48QL0sogxaGDhMlUo2gh14fHhcVmnOrrxNqrU9joX21Q0IUElm7DQkAOw==') !important; width: 34px !important; height: 21px !important; border: none !important; float:right; margin-top:-7px; margin-right:-10px; } </style>