Permalink | Leave a comment  »

SocialCaddy is *magic*! But how do we really do it?

For all us geeks out there, here is our most recent presentation during Erlang Factory 2010.

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Talking to IMAP services can be a source of frustration especially when each service implements parts of the protocol or add some secret ingredients to the sauce. One of the most popular services, as we all know is Gmail.

XOAuth

Gmail adds a lot of nice touches to the mail business. One of the latest additions is the use of XOAuth in order to authenticate against the service without any need of username and password. This can be especially handy if your app your app should read or just parse the headers of the users' emails but you do not want to save passwords, have extra UI for this purpose or let the user revoke the access at will. You can easily use xoauth in your webapp today using this small library.

Example code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

def connect_to_gmail(CREDENTIALS, email, oauth_token, oauth_token_secret):
    """
    Call this function to get an authenticated IMAP connection
    """
    consumer = OAuthEntity(CREDENTIALS[0], CREDENTIALS[1])
    access_token = OAuthEntity(oauth_token, oauth_token_secret)
    xoauth_string = GenerateXOauthString(
      consumer, access_token, email, 'imap',
      None, str(random.randrange(2**64 - 1)), str(int(time.time())))

    # connect to imap
    imap_conn = imaplib.IMAP4_SSL('imap.googlemail.com')
    #imap_conn.debug = 3
    imap_conn.authenticate('XOAUTH', lambda x: xoauth_string)
    imap_conn.select('INBOX')

Searching

By default when you use imap.select() you will be able to search within the INBOX folder but sometimes you may want to search in both Sent messages and Inbox. In order to do so you have to do give the following command:

imap_conn.select("[Gmail]/All Mail")

Fetching an email without setting it as SEEN

When you want to fetch an email in order to parse you usually do:

typ, msg_data = imap_conn.fetch(uid, 'RFC822')

which will do exactly what you want, but if the email is not read (SEEN) it will mark it as read which something you may not want if you are not building a Gmail client. If you simply want to parse/read the email then do the following

typ, msg_data = imap_conn.fetch(uid, '(BODY.PEEK[HEADER])')

Find the body message and decode it

This can be really tricky as there can be a lot of forwards and replies plus an awful lot of encodings. Usually we the encoding of an email is appended in the subject. The following gist can be to some help:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

def _process_body(pl):
	"""find the final payload (body) of an email"""
	if type(pl)==StringType:
		return pl.replace('\r\n>',' ').replace('\r\n',' ').replace('\n\n',' ')
	elif type(pl)==list:
		return _process_body(pl[0])
	elif type(pl)==InstanceType:
		return _process_body(pl._payload)
	else:
		print type(pl)

def _decode_body(subject,body):
	"""decode body if in base64"""
	try:		
		if re.search("(ISO-\d+-\d)", subject):
			iso = re.search("(ISO-\d+-\d)", subject).group(0)
			body = base64.b64decode(body).decode(iso)
			body = text.replace('\r\n>',' ').replace('\r\n',' ').replace('\n\n',' ')			
		return body.encode('utf-8') if type(body)==unicode else body
	except:
		return body.encode('utf-8') if type(body)==unicode else body

If you have any suggestions, corrections please feel free to fork the gists and let me know :)

Permalink | Leave a comment  »

The problem using Jetty (or Netty or Ring) is that after you start the process, your beloved REPL is "binded" to the server thus forcing you to kill the service, make a change, start the process again, refresh and do it again. Not fun at all. It's easy to run the process "in the background" and have the REPL available to make changes. 

Let's create a new project (using lein).

> lein new TinyService

Now, we need  a project.clj (so we can do a lein deps and have everything installed automagically)

1
2
3
4
5
6

(defproject TinyService "1.0.0-SNAPSHOT"
  :description "FIXME: write"
  :dependencies [[org.clojure/clojure "1.1.0"]
                 [org.clojure/clojure-contrib "1.1.0"]
		 [ring/ring "0.2.5"]]							  
  :dev-dependencies [[swank-clojure "1.2.0"]]

> lein deps

Edit core.clj with this:

1
2
3
4
5
6
7
8
9
10

(ns TinyService.core
  (:use ring.adapter.jetty))

(defn app [req]
  {:status  200
   :headers {"Content-Type" "text/html"}
   :body    "Hello world from TinyService!"})

(defn boot []
  (run-jetty #'app {:port 8080 :join? false}))

You can run the service in REPL by (boot) and then you'll still have access to REPL. You can change stuff, C+c C+c, refresh and voila! You can even connect to web server (to a JVM to be more exact) and make changes without the need for compilation. And all this from your local emacs!

Enjoy!

Permalink | Leave a comment  »

We've released a working of Amazon's API in Python here. You can even find similar items (Operation: SimilarityLookup). Just provide your access key and your secret key (you can get them here) and you are ready to go!

Permalink | Leave a comment  »

If you are running a web2py app on AppEngine it is matter of sanity to have AppStats logging your app 's performance. It is common practice to have performance monitor for all major web applications out there so that you know where to optimize your application. 

If you are on AppEngine the performance of your application is even more crucial as better performance means less costs and stability (as it is easy to hit the restrictions). 

If you have a web2py application it is very easy to this as web2py is first class wsgi citizen. Simply go to gaehander.py and modify the main function as in the following gist:

1 2 3 4 5

# gaehandler.py line:85 def main():     """Run the wsgi app"""     from appengine_config import webapp_add_wsgi_middleware     wsgiref.handlers.CGIHandler().run(webapp_add_wsgi_middleware(wsgiapp))

Then go to app.yaml and right after the handler definitions add these 2 lines:

1 2 3

handlers: - url: /stats.*   script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py

In the root directory of web2py create a new a file called appengine_config.py with the following content (thanx WritersEar for pointing out):

1 2 3 4

def webapp_add_wsgi_middleware(app):     from google.appengine.ext.appstats import recording     app = recording.appstats_wsgi_middleware(app)     return app

 

And you are good to go!

Permalink | Leave a comment  »

While working on the API of LinkedIn we had to a query in this format:

http://api.linkedin.com/v1/people/~/network?type=STAT&type=PICT&count=50&start=50

As usual, you get a 401 (forbidden) response. That was tough bug. Thank to LinkedIn forum I found out that in the python-oauth library you cannot have a query parameter with multiple values. Adam from the LinkedIn forum pointed out that have multiple values is valid according to the OAuth 1.0 spec and there is support in the python-oauth2 library.

If you want this functionality in python-oauth (OAuth 1.0) take a look at the following gist.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

From 1ca98a37b38a07387fc4344ce9df8db0a2885057 Mon Sep 17 00:00:00 2001
From: Panagiotis Papadopoulos <panos@socialcaddy.com>
Date: Tue, 29 Jun 2010 19:50:20 +0300
Subject: [PATCH] Added support for multiple query parameters with the same name

---
 oauth/oauth.py |   12 ++++++++++--
 1 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/oauth/oauth.py b/oauth/oauth.py
index 286de18..386636e 100644
--- a/oauth/oauth.py
+++ b/oauth/oauth.py
@@ -227,8 +227,16 @@ class OAuthRequest(object):
         except:
             pass
         # Escape key values before sorting.
-        key_values = [(escape(_utf8_str(k)), escape(_utf8_str(v))) \
-            for k,v in params.items()]
+        # Support multiple values for a query parameter as defined in OAuth spec
+        # at http://tools.ietf.org/html/rfc5849#section-3.4.1.3.2
+        key_values = []
+        for key, value in params.iteritems():
+            # 1.0a/9.1.1 states that kvp must be sorted by key, then by value,
+            # so we unpack sequence values into multiple items for sorting.
+            if hasattr(value, '__iter__'):
+                key_values.extend((key, item) for item in value)
+            else:
+                key_values.append((key, value))
         # Sort lexicographically, first after key, then after value.
         key_values.sort()
         # Combine key value pairs into a string.
-- 
1.6.4.2+GitX

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Today we are very excited: a couple of team members of HealthLeap , our favorite NYC startup and sister company of SocialCaddy, are working with us out from our Athens Office. Debugging had never been so much fun!

Have you heard of HealthLeap? If not, it's the easy way to make an appointment with your physician. That's how we do it! No more phone calls, no more planning 3 weeks in advance, no more tears. Like the opentable for doctors. 

There are a couple of other sites in the same space. But we really prefer HealthLeap. Why is it so much better? The HealthLeap team has the answer:

"Coming from families of physicians, we know how frustrating the business lives of private practitioners can be.  Physicians didn’t go to medical school to be in business – but with 87% of physicians in practices of 5 physicians or less, they are often times splitting their time with patients to essentially run a business.

Moreover, now that patients are moving en mass online, running a practice means creating a website, leveraging social media, and even offering the convenience online scheduling. We thought - why not make this all easier for physicians and other health practitioners?

As patients ourselves, we have to admit, we also got very excited about the idea of being able to find health practitioners and book our appointments online, even on the weekends or after a late night at work. We couldn’t wait to get started on HealthLeap!"

Permalink | Leave a comment  »

We're all embedded in vast social networks of friends, family, co-workers and more. We hate to admit but we are all clustered in herds and groups and tribes. We "like" and "dislike", are passionate or dull, and maintain the illusion that we solely decide how we feel and what we do. 

Nicholas Christakis tracks how a wide variety of traits -- from happiness to obesity -- can spread from person to person, showing how your location in the network might impact your life in ways you don't even know. His work has inspired us here at SocialCaddy and we wanted to share this talk with you. 

Permalink | Leave a comment  »

Oh Steve (he does remind me of my dad)!

This skit is from the guys over at 1938digital. It's hysterical and I wouldn't post it if jabs like that about the iPad (and the other Steve "the tea drinked") weren't that common here at SocialCaddy. We can't help it folks. Out team loves to hate (and love) all sorts of things. Have a wonderful Sunday :-)

Permalink | Leave a comment  »

Permalink | Leave a comment  »

We want SocialCaddy to send HTML emails with daily digests to our users. Well there is nothing fancy about it but we spent one day trying to do that on Google App Engine.

We use the Python framework web2py. After digging with the web2py source code we found a tiny bug that prevented us from sending HTML emails from App Engine. If you face a similar problem then just go to gluon/tools.py and replace line 437 with the following code:

result = mail.send_mail(sender=self.settings.sender, to=to, subject=subject, body=text, html=html)

We hope that no one else will spend 5 hours on such a tiny bug :)

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Permalink | Leave a comment  »

The first thing people ask me when I tell them I am in start-up is "how it feels to be in a start-up?". I usually tell them that it feels exactly like being in love. And it's true. And that's the bad part.

So, you found a person you love (and hope she/he loves you back) and it's the best thing that ever happened to you (at least this week). You feel invincible and that you can do anything! Well, this is "Phase 1" in the start-up world!

Phase 1: Exaggeration or "I am building the new Facebook/Google!"

Every single entrepreneur thinks that his idea is the best idea in the world. And that's good. If you don't believe it yourself, why someone else will? The world falls apart when your idea gets shot down. But you don't care. Because you are in love. In love with your idea. And that feels like nitro in your blood stream. 

In order to succeed in the Start-up world, you reaaaally need to love your product. You need to crave for perfection, skip meals, sleep, baths, sex and everything else that might please you. And you don't care. Because you are with your dream girl. And that's what only matters.

Yeap, Start-ups are like oxygen :)

Coming up next:

Phase 2: Fear

Phase 3: Euphoria

Phase X: Mood Swings

Permalink | Leave a comment  »

UPDATE: That's an event worth checking out if you are considering making the transition: Working at a startup . 

(summary: If you're like a lot of programmers you may have considered one day joining a startup. But the prospect probably seems a bit mystifying. Most people know what the deal is with working for a big company. What's the deal with working for a startup?

Work at a Startup is a special evening event designed to explain that. How can you tell whether a startup would be good to work for? How much salary and equity should you expect from startups at different stages? What's the work like at different types of startups? Are there any danger signs you should watch for? We'll answer all those questions.)

Permalink | Leave a comment  »

Permalink | Leave a comment  »