diff options
Diffstat (limited to 'build/blog/2015-12-27-lesson-on-magic-method-access-of-python-new-style-classes-from-my-failed-python3-port-of-tomorrow.html')
| -rw-r--r-- | build/blog/2015-12-27-lesson-on-magic-method-access-of-python-new-style-classes-from-my-failed-python3-port-of-tomorrow.html | 118 |
1 files changed, 118 insertions, 0 deletions
diff --git a/build/blog/2015-12-27-lesson-on-magic-method-access-of-python-new-style-classes-from-my-failed-python3-port-of-tomorrow.html b/build/blog/2015-12-27-lesson-on-magic-method-access-of-python-new-style-classes-from-my-failed-python3-port-of-tomorrow.html new file mode 100644 index 00000000..839ce5d5 --- /dev/null +++ b/build/blog/2015-12-27-lesson-on-magic-method-access-of-python-new-style-classes-from-my-failed-python3-port-of-tomorrow.html @@ -0,0 +1,118 @@ +<!DOCTYPE html> +<html> +<head> +<meta charset="utf-8"/> +<meta content="pandoc" name="generator"/> +<meta content="Zhiming Wang" name="author"/> +<meta content="2015-12-27T16:47:05-08:00" name="date"/> +<title>Lesson on magic method access of Python new-style classes (from my failed Python3 port of Tomorrow)</title> +<link href="/img/apple-touch-icon-152.png" rel="apple-touch-icon-precomposed"/> +<meta content="#FFFFFF" name="msapplication-TileColor"/> +<meta content="/img/favicon-144.png" name="msapplication-TileImage"/> +<meta content="width=device-width, initial-scale=1" name="viewport"/> +<link href="/css/normalize.min.css" media="all" rel="stylesheet" type="text/css"/> +<link href="/css/theme.css" media="all" rel="stylesheet" type="text/css"/> +<link href="/css/highlight.css" media="all" rel="stylesheet" type="text/css"/> +</head> +<body> +<div id="archival-notice">This blog has been archived.<br/>Visit my home page at <a href="https://zhimingwang.org">zhimingwang.org</a>.</div> +<nav class="nav"> +<a class="nav-icon" href="/" title="Home"><!--blog icon--></a> +<a class="nav-title" href="/"><!--blog title--></a> +<a class="nav-author" href="https://github.com/zmwangx" target="_blank"><!--blog author--></a> +</nav> +<article class="content"> +<header class="article-header"> +<h1 class="article-title">Lesson on magic method access of Python new-style classes (from my failed Python3 port of Tomorrow)</h1> +<div class="article-metadata"> +<time class="article-timestamp" datetime="2015-12-27T16:47:05-08:00">December 27, 2015</time> +</div> +</header> +<p>I know the title is formidably long, but I can't find something more accurate (and my homegrown mini CMS doesn't support subtitle), so please bear with me.</p> +<p>So, I have <a href="https://github.com/madisonmay/Tomorrow">madisonmay/Tomorrow</a> — "magic decorator syntax for asynchronous code in Python 2.7" — bookmarked for a long time<a class="footnoteRef" href="#fn1" id="fnref1"><sup>1</sup></a> without ever trying it, because I simply don't write Python 2 code any more (except when I try to maintain compatibililty). I felt kind of strange that a ~50-line project with ~1000 stars on GitHub hasn't been ported to Python 3 already, so I gave it a shot just now.</p> +<p>I thought it would be easy:</p> +<ol style="list-style-type: decimal"> +<li>Modernize the old-style class <code>Tomorrow</code>;</li> +<li>Replace <code>__getattr__</code> with <code>__getattribute__</code> for unconditional attribute routing, then make a few exceptions to prevent infinite recursion;</li> +<li><code>2to3</code> test cases;</li> +<li>Make meta changes, like removing the <code>futures</code> dependency.</li> +</ol> +<p>However, after doing 1–3, I ran the tests, and out of the five test cases, three failed and one errored. I tried to isolate the problem, and ended up with the following piece of proof-of-concept:</p> +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="kw">class</span> PassThrough(<span class="bu">object</span>): + + <span class="kw">def</span> <span class="fu">__init__</span>(<span class="va">self</span>, obj): + <span class="va">self</span>._obj <span class="op">=</span> obj + + <span class="kw">def</span> <span class="fu">__getattribute__</span>(<span class="va">self</span>, name): + <span class="cf">if</span> name <span class="op">==</span> <span class="st">"_obj"</span>: + <span class="cf">return</span> <span class="bu">object</span>.<span class="fu">__getattribute__</span>(<span class="va">self</span>, name) + <span class="bu">print</span>(<span class="st">"Accessing '</span><span class="sc">%s</span><span class="st">'"</span> <span class="op">%</span> name) + <span class="cf">return</span> <span class="va">self</span>._obj.<span class="fu">__getattribute__</span>(name)</code></pre></div> +<p>This snippet is valid in both Python 2.7 and Python 3, but here's the surprise:</p> +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="op">>>></span> g <span class="op">=</span> PassThrough(<span class="dv">0</span>) +<span class="op">>>></span> <span class="bu">print</span>(g) +<span class="op"><</span>__main__.PassThrough <span class="bu">object</span> at <span class="bn">0x10c662e48</span><span class="op">></span> +<span class="op">>>></span> <span class="bu">str</span>(g) +<span class="co">'<__main__.PassThrough object at 0x10c662e48>'</span> +<span class="op">>>></span> <span class="bu">hasattr</span>(g, <span class="st">'__str__'</span>) +Accessing <span class="st">'__str__'</span> +<span class="va">True</span> +<span class="op">>>></span> g.<span class="fu">__str__</span>() +Accessing <span class="st">'__str__'</span> +<span class="co">'0'</span></code></pre></div> +<p>In addition, here's what happens if you try to "pass through" a function:</p> +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="op">>>></span> <span class="kw">def</span> f(): <span class="cf">return</span> <span class="va">True</span> +<span class="op">>>></span> g <span class="op">=</span> PassThrough(f) +<span class="op">>>></span> g() +Accessing <span class="st">'__class__'</span> +Accessing <span class="st">'__class__'</span> +Traceback (most recent call last): + File <span class="st">"<ipython-input-6-d65ffd94a45c>"</span>, line <span class="dv">1</span>, <span class="kw">in</span> <span class="op"><</span>module<span class="op">></span> + g() +<span class="pp">TypeError</span>: <span class="st">'PassThrough'</span> <span class="bu">object</span> <span class="kw">is</span> <span class="kw">not</span> <span class="bu">callable</span> + +<span class="op">>>></span> <span class="bu">callable</span>(g) +<span class="va">False</span> +<span class="op">>>></span> <span class="bu">hasattr</span>(g, <span class="st">'__call__'</span>) +Accessing <span class="st">'__call__'</span> +<span class="va">True</span> +<span class="op">>>></span> g.<span class="fu">__call__</span>() +Accessing <span class="st">'__call__'</span> +<span class="va">True</span></code></pre></div> +<p>As you can tell, although <code>__str__</code> or <code>__call__</code> may have been implemented through <code>__getattribute__</code>, and <code>hasattr</code> (which in turn depends on <code>getattr</code>) has no trouble finding them, they are not picked up by <code>str</code> or function call <code>(...)</code>. At this point, one would suspect that this is due to <code>str</code> or function call only looking at the class instance's <code>__dict__</code>. Compare this to the behavior of an old-style class:</p> +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="kw">class</span> PassThrough(): + + <span class="kw">def</span> <span class="fu">__init__</span>(<span class="va">self</span>, obj): + <span class="va">self</span>._obj <span class="op">=</span> obj + + <span class="kw">def</span> <span class="fu">__getattr__</span>(<span class="va">self</span>, name): + <span class="bu">print</span>(<span class="st">"Acessing '</span><span class="sc">%s</span><span class="st">'"</span> <span class="op">%</span> name) + <span class="cf">return</span> <span class="va">self</span>._obj.<span class="fu">__getattribute__</span>(name)</code></pre></div> +<p>Now:</p> +<div class="sourceCode"><pre class="sourceCode python"><code class="sourceCode python"><span class="op">>>></span> g <span class="op">=</span> PassThrough(<span class="dv">0</span>) +<span class="op">>>></span> <span class="bu">print</span>(g) +Acessing <span class="st">'__str__'</span> +<span class="dv">0</span> +<span class="op">>>></span> <span class="kw">def</span> f(): <span class="cf">return</span> <span class="va">True</span> +<span class="op">>>></span> g <span class="op">=</span> PassThrough(f) +<span class="op">>>></span> g() +Acessing <span class="st">'__call__'</span> +<span class="va">True</span></code></pre></div> +<p>Note that magic method access is always routed through <code>__getattr__</code>.</p> +<p>After some digging, my suspicion was confirmed: indeed, for new-style classes, rather than invoking <code>__getattribute__</code>, the Python interpreter only looks for magic methods in <code>__dict__</code>. But is there a workaround for implementing something like the <code>PassThrough</code> class above? There's a <a href="http://stackoverflow.com/a/9059858/">nice answer</a> on StackOverflow that uses a metaclass to "automatically add proxies for magic methods at the time of class creation", to quote the author. However, the thing about Tomorrow is that <em>we don't have the result and don't know whatever magic methods it might have at class creation</em> — after all, Python isn't a statically typed language. It is possible for programmers to offer hints, but then Tomorrow won't be as elegant and magical anymore. Therefore, unfortunately enough, Tomorrow isn't portable to Python 3 — at least not without a substantial hack that's beyond my knowledge, or a complete overhaul of its logic (haven't thought about that).</p> +<div class="footnotes"> +<hr/> +<ol> +<li id="fn1"><p>Pretty much since the beginning, I believe (the <a href="https://github.com/madisonmay/Tomorrow/commit/22a53dfbcf9b516ecd1770eeca9fcf1720271240">initial commit</a> was from July 24 of this year). I don't remember how I came accross it though.<a class="footnotes-backlink" href="#fnref1">↩︎</a></p></li> +</ol> +</div> +</article> +<hr class="content-separator"/> +<footer class="footer"> +<span class="rfooter"> +<a class="rss-icon" href="/rss.xml" target="_blank" title="RSS feed"><!--RSS feed icon--></a><a class="atom-icon" href="/atom.xml" target="_blank" title="Atom feed"><!--Atom feed icon--></a><a class="cc-icon" href="https://creativecommons.org/licenses/by/4.0/" target="_blank" title="Released under the Creative Commons Attribution 4.0 International license."><!--CC icon--></a> +<a href="https://github.com/zmwangx" target="_blank">Zhiming Wang</a> +</span> +</footer> +</body> +</html> |