171
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
170
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
169
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
168
|
|
|
Ken VanDine |
11 years ago
|
|
|
167
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
166
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
165
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
164
|
|
* Stop deduplicating messages across protocols, simplifying model schema (LP: #1156941) * Add schema columns for latitude, longitude, and location name. * Fix 'likes' column from gdouble to guint64. * Add geotagging support from foursquare, facebook, flickr. * Implement since= for Facebook, reducing bandwidth usage. * Automatically prepend the required @mention to Twitter replies (LP: #1156829) * Automatically linkify URLs that get published to the model. * Fix the publishing of Facebook Stories (LP: #1155785). Fixes: https://bugs.launchpad.net/bugs/1135038, https://bugs.launchpad.net/bugs/1155785, https://bugs.launchpad.net/bugs/1156829, https://bugs.launchpad.net/bugs/1156941, https://bugs.launchpad.net/bugs/1156979.
Approved by PS Jenkins bot, Ken VanDine.
|
Robert Bruce Park |
11 years ago
|
|
|
163
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
162
|
|
|
Martin Pitt |
11 years ago
|
|
|
161
|
|
|
Automatic PS uploade... |
11 years ago
|
|
|
160
|
|
Start using since_id= on Twitter API requests. (LP: #1152417)
This was accomplished by implementing two new classes, and ended up simplifying some of the RateLimiter code as a side effect.
The first new class is called JsonCache. It is a subclass of dict, which attempts to populate it's initial state by reading in a json text file at a configurable location, and also adds a new "write()" method that dumps the json back out to the same location. This class was a generalization of what we were already doing inside the RateLimiter, so it should not be considered a "new feature" if we are going to butt heads with today's feature freeze.
The second new class is a subclass of JsonCache, which enforces that:
A) keys may not contain slashes, to avoid it getting polluted with every search term ever searched for, or every message that's ever been replied to ever, it only observes the values of the "main" streams, such as "messages", "mentions" and "private", although those values are not hardcoded so it's flexible to adapt to new streams in the future.
B) values must be ints (tweet_ids), and values can only be incremented. This is so that we can easily just throw every observed tweet_id into the cache, and it only records the largest (newest) one.
The end result is that we now have two new files located at ~/.cache/friends/twitter_ids.json and ~/.cache/friends/identica_ids.json which track the newest tweet_id that we have ever seen for each of the streams that we publish to. These values are then consulted to form the since_id= argument to several of Twitter's API endpoints, which solves bug #1152417.
As an added bonus, this also greatly reduces our network usage because we are no longer redownloading duplicate messages over and over, so if there are no new messages, Twitter is now returning an empty list of Tweets rather than a large list of stale tweets.
This commit includes full test coverage for all new code. Fixes: https://bugs.launchpad.net/bugs/1152417.
Approved by Ken VanDine, PS Jenkins bot.
|
Robert Bruce Park |
11 years ago
|
|
|
159
|
|
|
Ken VanDine |
11 years ago
|
|
|
158
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
157
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
156
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
155
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
154
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
153
|
|
|
Robert Bruce Park |
11 years ago
|
|
|
152
|
|
|
Robert Bruce Park |
11 years ago
|
|
|