Free AI web copilot to create summaries, insights and extended knowledge, download it at here
4324
Abstract
span> <span class="hljs-params">(c *calculator)</span></span> Calculate(num... <span class="hljs-type">int</span>) {
ans := <span class="hljs-number">0</span>
<span class="hljs-keyword">for</span> _, n := <span class="hljs-keyword">range</span> num {
ans += n
}
fmt.Println(ans)
}
<span class="hljs-keyword">var</span> Calculator calculator</pre></div><p id="c1d0">We exposed a struct which implements the method Calculate. This is required for the discovery and registration of the plugin. Now to compile this and make it a shared-object we need to build the library as a plugin.</p><div id="b6b4"><pre><span class="hljs-attribute">go build -o -buildmode</span>=plugin .</pre></div><p id="198a">After this, we will have a shared object file(in our case “<i>add.so</i>”) which we can dynamically load at runtime in our main application.</p><figure id="5129"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*HEBnzWVkvUP0JjhOsfOH7Q.png"><figcaption>updated file structure of the library</figcaption></figure><p id="5408">Now to dynamically load the library into our main application at run time we need use the plugin system of Golang.</p><div id="0960"><pre><span class="hljs-keyword">import</span> (
<span class="hljs-string">"fmt"</span>
<span class="hljs-string">"os"</span>
<span class="hljs-string">"plugin"</span>
)
<span class="hljs-keyword">type</span> Calculator <span class="hljs-keyword">interface</span> {
Calculate(num... <span class="hljs-type">int</span>)
}
<span class="hljs-function"><span class="hljs-keyword">func</span> <span class="hljs-title">main</span><span class="hljs-params">()</span></span> {
calculatorPlugin, err := plugin.Open(<span class="hljs-string">"/abs/path/to/shared/object/file"</span>)
<span class="hljs-keyword">if</span> err != <span class="hljs-literal">nil</span> {
fmt.Println(<span class="hljs-string">"error while opening shared object file"</span>)
os.Exit(<span class="hljs-number">1</span>)
}
symCalculator, err := calculatorPlugin.Lookup(<span class="hljs-string">"Calculator"</span>)
<span class="hljs-keyword">if</span> err != <span class="hljs-literal">nil</span> {
fmt.Println(<span class="hljs-string">"error while lookup"</span>)
os.Exit(<span class="hljs-number">1</span>)
}
<span class="hljs-keyword">var</span> calculator Calculator
calculator, ok := symCalculator.(Calculator)
<span class="hljs-keyword">if</span> !ok {
fmt.Println(<span class="hljs-string">"unexpected type from module symbol"</span>)
os.Exit(<span class="hljs-number">1</span>)
}
calculator.Calculate(<span class="hljs-number">3</span>,<span class="hljs-number">4</span>)
}</pre></div><p id="f4f9">In this case if we are making any changes to library, we don’t need to re-compile our main application. We only need to recompile the library’s shared object and rerun our main application.</p><p id="a32d">Due to the different characteristics, the advantages and disadvantages of static and dynamic libraries are also obvious; binaries that rely only on static libraries and are generated by static linking can be executed independently because they contain all the dependencies, but the compilation result is also larger. Dynamic libraries can be shared among multiple executables, which can reduce the memory footprint, and their linking process is often triggered during loading or running, so they can contain some modules that can be hot-plugged and reduce the memory footprint. Compiling binaries using static linking has very obvious deployment advantages, and the final compiled product will run directly on most machines. The deployment benefits of static linking are far more important than the lower memory footprint, that’s why Golang uses static linking as the default linking method.</p><h1 id="d290">Issues with Dynamic-linking (shared library plugins) in Go</h1><p id="91cd">Plugins using shared libraries and the plugin package work well for Golang, as the previous section demonstrates. However, this approach also has some serious downsides. The most important downside is that Golang is very picky about keeping the main application and the shared libraries it loads compatible.</p><p id="285c">As an experiment, try using different versions of a common d
Options
ependency in the plugin application and the main application, rebuild the main application and run it. Most likely you’ll get this error:</p><div id="443c"><pre><span class="hljs-comment">"plugin was built with a different version of package XXX"</span></pre></div><figure id="d928"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/0*ow5zqQqWHmQoZqaC.gif"><figcaption></figcaption></figure><p id="35aa">The reason for this is that Golang wants all the versions of all packages in the main application and plugins to match exactly. It’s clear what motivates this: safety.</p><p id="7b0e">Consider C and C++ as counter-examples. In these languages, an application can load a shared library with dlopen and subsequently use dlsym to obtain symbols from it. dlsym is extremely weakly typed; it takes a symbol name and returns a void*. It’s up to the user to cast this to a concrete function type. If the function type changes because of a version update, the result can very likely be some sort of segmentation fault or even memory corruption.</p><p id="80a8">Since Golang relies on shared libraries from plugins, it has the same inherent safety issues. It tries to protect programmers from shooting themselves in the foot by ensuring that the application has been built with the same versions of packages as all its plugins. This helps avoid mismatch. In addition, the version of the Golang compiler used to build the application and plugins must match exactly.</p><p id="74b9">However, this protection comes with downsides — making developing plugins somewhat cumbersome. Having to rebuild all plugins whenever any common packages change — even in ways that don’t affect the plugin interface — is a heavy burden. Especially, considering that by their very nature plugins are typically developed separately from the main application; they may live in separate repositories, have separate release cadences etc.</p><h1 id="bfc1">Alternative approaches to shared library plugins in Golang</h1><p id="c642">Given that the plugin package was only added to Go in <a href="https://golang.org/doc/go1.8">version 1.8</a> and the limitation described previously, it’s not surprising that the Go ecosystem saw the emergence of alternative plugin approaches.</p><p id="87e5">One of the innovative approaches involves plugins via RPC. In this method instead of loading the plugins into the host process we load them in a separate process which then communicates to host via RPC or just TCP on localhost. It has several important upsides:</p><ul><li>Isolation: crash in a plugin does not bring the whole application down.</li><li>Interoperability between languages: if RPC is the interface, do you care what language the plugin is written in?</li><li>Distribution: if plugins interface via the network, we can easily distribute them to run on different machines for gains in performance, reliability, and so on.</li></ul><p id="f78c">Moreover, Golang makes this particularly easy by having a fairly capable RPC package right in the standard library: net/rpc.</p><p id="4495">One of the most widely used RPC-based plugin systems is <a href="https://github.com/hashicorp/go-plugin">hashicorp/go-plugin</a>. Hashicorp is well known for creating great Golang software, and apparently, they use go-plugin for many of their systems, so it’s battle-tested (though its documentation could be better 😛)</p><p id="1a23">Golang-plugin runs on top of net/rpc but also supports gRPC. Advanced RPC protocols like gRPC are well suitable for plugins because they include versioning out-of-the-box, tackling the difficult interoperability problem between different versions of plugins vs. the main application.</p><p id="c8a4">However, this is also<b> not a perfect solution </b>as we talk about the fourth fundamental of plugin systems- “<b>Extension APIs</b>”. In a complex system, a plugin might make lot of Extension APIs calls which will end up increasing the latency if we are making network calls via RPC or TCP.</p><h1 id="1c76">Conclusion</h1><p id="03fb">Golang still has a long way to go when it comes to Dynamic-linking (shared library plugins). No solution discussed in this blog can be considered perfect. One needs to consider the pros and cons of each solution available as per their specific requirements.</p></article></body>