Bosco Curtu

Archive for the ‘django/python’ Category

Hoy los amigos de Festuc están actualizando su web con una nueva versión hecha en Django. Enhorabuena festuceros! Nos habéis ganado en la carrera de ser la primera (gran) web hecha en Django en Cataluña. A nosotros nos falta nada, un par de semanas.

Festuc y dooplan tienen mucho en común, además de que ambas plataformas serán Django, compartiremos un mismo objetivo, convertirnos en referencia del ocio. Sin embargo el enfoque es ligeramente diferente. Mientras ellos van a por un target de twenty something, fiesteros, aves nocturnas…, nosotros desde dooplan intentaremos posicionarnos en un rango de edad superior y con una oferta de ocio más variada. No queremos centrarnos en fiestas exclusivamente, queremos también llegar a esos que disfrutan con una buena exposición, una conferencia, una feria, con sus hijos, con su pareja… Un público diferente.

Jordi me ha pasado un link a Consensus, una sencilla librería python para filtrado colaborativo. Es extremadamente sencilla (que no simple) por lo que se puede entender facilmente la filosofía detras. Interesante.

Clustering users and items is an interesting technique when you work with big data sets. Due to the complexity of the calculations, and RAM consumption, it’s really useful to work with smaller sets. When you cluster users (or items) what you get are groups of similar users, so in addtion to have a smaller data set, the opinions of this users will be closer among them, and the recommendations may improve.

django-recommender implements 2 methods for clustering:

• def cluster_users(self, users, items, cluster_count=2):
• def cluster_items(self, users, items, cluster_count=2):

Both methods use django-voting for its calculations, because it must be consistent with the other methods. If we use a criteria for getting people similarity, we must respect it when grouping people in clusters.

The result of both methods is a list of sublists, containing the object ids of the elements in each cluster. For instance, you have 10 user (ids from 1 to 10), and want to have 4 clusters, a possible result may be:

user_cluster = [[2,9,1],[3],[5,10,8,6],[4,7]]

Bear in mind the result can vary, because there is a random conponent in first centroids calculation, and more than one solution is possible for the problem.

He hecho algunas pequeñas mejoras en el django-recommender. La más importante ha sido el cambio de la librería statistics (una cosa rara de unos japos) por la estandar python stats. Por supuesto, está incluida en el proyecto, por lo que no hace falta instalarla en local.

Por otro lado, he refactorizado el método get_content_based_recs, para que funcionara de la misma manera que con los de filtrado colaborativo. Se le pasan los objetos a comparar y el manager se busca la vida (básicamente, obtiene sus atributos usando django-tagging).

Además, he desacoplado un poco el código, dejando en el managers.py los métodos que se suponen que se deben usar en la aplicación, y he añadido un utils.py con la lógica más del recomendador, los cálculos y así.

Por último, he añadido un par de pelis más al juego de test para que hubiera un poco más de resultados variados.

Content Based filtering technique is the classical approach for recommendations. This technique has been used before the internet recommenders, because it’s just the comparison between two elements. Currently it is very typical in matching recommenders, like meetic.com or match.com.

However, although the results aren’t any impressive it can be useful in several scenarios: for matching elements in a non-voting system (meetic again), to create alarms, or having a second opinion to weight and  complete your collaborative filtered recommendations.

The first scenario is obvious. The second one can be useful for fuzzy or rough searches, and the third scenario is usefull to weight the collective recommendation with what the user really likes.

For django-recommender, the usage is based on tags. It uses django-tagging to tag any element of the system. The algorithm just gets the list of tags of a user and the list of tags for all the other items of the system, and it calculates the distance value using a modified version of Tanamoto algorithm. Tanamoto is too demanding, because it checks the position of very element of the arrays. In this version, the order is not important. For systems with too much tag dispersion, tanamoto may return poor results, so it could be interesting on simpler tag comparision (f.i., if there is a coincidence, return 0.5, if there are more than one, return 0,7…)

To use this technique you need to install django-tagging application, and to tag the user and the elements you want to recommend. Then ,just invoke:

Recommender.objects.get_content_based_recs(user_tags,
                                         item_tag_matrix)

And you will get a paired list containg: [(rec_value, item), (rec_value,item)...], where 0.0<rec_value<1.0, and 1.0 is best.

Por fin han publicado mi django-recommender en django pluggables! Ahora solo queda usarlo! La semana que viene haré un upgrade del proyecto para introducir recomendaciones basadas en tags (usando django-tagging plugable)

django-recommender is my new pluggable for having recommendation functionality in any django project. The only requierement is to have also django-voting (and to use it…)

The philosophy behind is to be able to make recommendations based on the user’s experience compared with the other user’s one. So instead of getting only the most popular items on the system, the user can also discover the items that can interest him more. The algorithms are based on Toby Segaran’s book “Programming Collective Intelligence“.

It’s pretty easy to use, you just need to use django-voting for vote the model you want to recommend, and the call the available functions on the RecommenderManager class.

For this first release I have included the next functions:

• get_best_items_for_user: This function will return a sorted list of pairs (value, item) containing the recommended items for a given user, based on the votes of the users.
• get_similar_users: This function will tell us which are the most similar users, based on the historical votes for similar items.
• get_best_users_for_item: This function will return the best candidates that may like a given item, based on votes.
• get_similar_items: The closest items, based on the votes of the users.

With this first set of functions we can answer most of the questions for a typicall social net, Which one is the best item for me? Who other users may I be interested on? Which users may be the best potential consumers for a given item? Which other items are similar to this one?

A simple snippet to show a bit of code. For better information, have a look at tests.py file:

        users = User.objects.all()
        movies = Movie.objects.all()
        user = User.objects.get(id=4)
        items = Recommender.objects.get_best_items_for_user(user,
                                                users, movies)

The items variable contains a list with the next signature:

[(0.5, <TestItem: Animal House>),
 (0.25, <TestItem: Casablanca>)]

That means you may like Animal House with a 0.5 value (1 is max), and Casablanca with 0.25.

La comunidad Django nos ha dado tanto, tantísimo, que sin ella sería imposible hacer un proyecto como en el que estamos currando. Por eso, en cuanto nos es posible, en dooplan.com intentamos devolver algo, una minúscula parte.

Hoy he tenido la necesidad de mejorar django-pagination, un maravilloso plugable para paginar cualquier lista de forma super transparente. En mi caso necesitaba paginar una lista dentro de un div generado dinámicamente con ajax, es decir, al pulsar “next” no quiero que se recargue toda la página, solo el div. En el post anterior explico como hacerlo.

Por otra parte, Hector tiene casi listo un plugable que hemos necesitado, un framework de privacidad que permite definir la privacidad de acceso y modificación a un objeto cualquiera. En cuanto lo generalice del todo y termine de documentarlo, lo publicará en google-code.

Pues eso, poquito a poco y entre todos construimos esta gran herramienta.

I’ve been using django-pagination for a while. However I have needed a special feature: to paginate in a dynamic (ajax) loaded div.

I have patched django-pagination for this issue. I’ve attached 3 files with the patch:

• pagination_tags.py, with the applied changes. Replace the django-pagination templatetag file with this one.
• pagination_tags.py.patch, the patch with these changes
• ajax_pagination.html, a new template for this ajax pagination. Just put it in the django-pagination tamplates folder.

To use this tag, just replace your {% paginate %} tag by the new {% ajax_paginate %}

The only requirements are to use jQuery and to have in the request context two new values:

• dynamic_div: The id of the div where the dynamic content is loaded
• ref_url: The ajax url to load the paginated elements of the div

As an example:

• An html with dynamic content:

 <div id="dynamic_tab">&nbsp;</div>
  <script>
 jQuery().ready(function() {
   var tab=jQuery("#dynamic_tab");
   tab.load("/an_ajax_list_view/") 
  });
 </script>

• The views.py:

def an_ajax_list_view(request):
 ...
  context = RequestContext(request)
  context['dynamic_div'] = 'dynamic_tab'
  context['ref_url'] = reverse('an_ajax_list-view',args=[])
 ... 
  return HttpResponse(render_to_string('my_dynamic_div_template.html',
                          context),mimetype='text/plain')

• And my_dynamic_div_template.html, with the paginated list:

 ...
{% autopaginate list 6 %}
 {% for element in list %}
  {{ element }}
 {% endfor %}
{% ajax_paginate %}
...

I hope it’s usefull for you, too. If you need more details, just tell it.