~sambuddhabasu1/mailman/fix_mailman_run

1

Mailman - Extensibility and Python

2

(was Mailman and the Potency of a Python-Powered MLM)

3

4

.+ Motivation: Internet dynamicism calls for adaptable MLMs

5

6

From the early days of the ARPAnet to today, email and Mailing List

7

Management (MLM) systems have played a crucial role in the formation

8

and conduct of communities on the internet. With the internet's

9

profound dynamicism, the form of those communities has dramatic

10

potential for continued growth and evolution. The extent to which

11

MLMs are versatile and adaptable is the extent to which they can

12

accomodate, and even foster, effective new forms of community

13

organization. Mailman is specifically designed for such evolution,

14

and one of the contributing features is its basis in Python.

15

16

In this paper we will look at customizability and extensibility in

17

Mailman, considering how the systems design and, in particular, the

18

implementation language, Python, factor into it. We examine a

19

continuum of adaptability in the system, ranging from simple,

20

non-intrusive option-setting customizations, to incorporation of entire

21

new subsystems, like a mail/news gateway, at the lowest level.

22

23

. - Why Python?

24

25

Python is particularly well suited to to implementing an extensive and

26

changing system.

27

28

Python code tends to be emminently readable, due to its regular,

29

uncluttered syntax and promotion of an uncluttered layout, and due to

30

its emphasis on cogent, high-level data types and control constructs.

31

This combination of clean syntax and cogent semantics aids the

32

programmer, all the more in the process of changing existing code.

33

34

The language is dynamic in many respects. It has the immediacy of

35

interpretation and dynamic typing, enabling an interactive

36

change/test/modify cycle. Everything is an first class object, with

37

introspection, enabling interaction with and programmatic handling of

38

just about everything in the language.

39

40

Automatic memory management, strict typing, comprehensive provisions

41

for encapsulation - including comprehensive object-oriented

42

programming and storage modules - all facilitate the development

43

process and reduce programming pitfalls.

44

45

Why is all this important in a contemporary MLM? A good MLM needs to

46

accomodate continuing development and change. The internet community

47

is growing, and the infrastructures by which organizes itself are

48

evolving. A good MLM will foster that evolution by growing with it.

49

50

Another reason for extensibility's importance has to do with the

51

expertise of mailman administrators. These administrators are

52

typically near enough to the end-users to get clear impressions of

53

their needs. Also, more than occasionally they are technically savvy

54

enough to be able to implement improvements to accomodate those needs

55

- provided the system doesn't present too high a threshold of

56

comprehenshion. Here is a prime opportunity for exploiting the

57

Bazaar-style of system development [klm: cite esr's paper], enabling

58

the facilitators of the medium, themselves, to guide its development,

59

so it can be closely tailored to the needs of the community using it.

60

61

Finally, most aspects of an MLM do not require the kind of speed

62

optimizations which force change-impeding hardening of system.

63

Performance critical aspects, like mail delivery to large numbers of

64

users, is generally the purview of the underlying Mail Transport

65

Agents (MTAs), rather than the MLM. Large capacities can impose some

66

specialized performance demands on the MLM, of course. The speciality

67

of those demands, however, enables isolating the optimizations to

68

select components, and Python's compiled-language extensibility

69

enables hardening those specific components as needed, isolating the

70

rigidity to the particular subsystems that need it.

71

72

. - Ranges of adaptability

73

74

The least intrusive provisions for changing a system's operation

75

involve configuration options, programmed in for anticipated needs.

76

By organizing and formally structuring the options mechanism, the

77

programmer simplifies the eventual process of incorporating new

78

options, minimizing their impact on the program's structure.

79

80

Similarly, by formulating a procedural template for certain routine

81

tasks, hooks can be implemented so that changes to the program only

82

involve supplying new code that implements a plug-in function which

83

satisfies the template, avoiding any change to the core program.

84

85

A programmer can only anticipate so much in their design for a system.

86

At some point, particularly in a dynamic world, deeper changes to

87

existing code, and even to the organization of the code, will often be

88

necessary. Though the designer cannot provide for all eventualities,

89

good fundamental design and clean implementation can facilitate the

90

program's evolution.

91

92

Mailman provides some extensive customization facilities, including

93

parameters for many of the operational behaviors of a maillist, or of a

94

subscriber's membership, and also mechanisms for customizing, via the

95

web, the layout of the web pages associated with a maillist. Mailman

96

is also organized to facilitate changes and additions to the program,

97

capitalizing on Python's object-oriented programming facilities and

98

modularity.

99

100

Below we will first introduce Mailman's architecture, and then examine

101

in greater detail some changes that have recently been made to

102

mailman's, particularly including the customization mechanisms, in

103

order to expose the ways that use of Python has facilitated extension

104

and evolution of the system.

105

106

.+ An introduction to Mailman's architecture

107

108

. - MailList class and mixins

109

110

[klm: i'm not doing this section, but here's my read on the mixin vs

111

delegate question.]

112

113

The MailList class is composed by inheritance from of a number of

114

task-oriented component classes, as mixins. The task oriented

115

components textually gather together code related to a particular

116

functionality, and include initialization of task-specific data, while

117

mixin composition enables easy sharing of methods and data throughout

118

the maillist object. This avoids the need to explicitly identify

119

components in order to use their data and methods. Also, methods

120

within what would be delegate classes are automatically available

121

externally, avoiding the need for redundant mediator methods.

122

123

This arrangement has a potential drawback, lack of

124

compartmentalization, which increases the possibility of name

125

collisions across components. Also, the coder cannot as easily

126

identify which class a method comes from, needing to search (eg, grep)

127

where a particular method or variable is defined. In practice we have

128

not experienced collisions, and in general have enjoyed the coding

129

ease of sharing methods and data.

130

131

. : Configuration variables

132

. . default values and site configuration - Defaults.py vs mm_cfg.py

133

. . Option declarations

134

. ; Persistent storage

135

. ; User interface - maillist administrator

136

. . Pending issues?

137

138

. : Filtering of postings

139

. . All postings of dubious character are held for admin approval

140

. . Dubious character is configurable according to sections of a

141

filter routine

142

. . New filtering mechanisms, eg spam protection, have been added

143

. . Probably better as a hook-list of filters, which are passed the

144

list object, to obtain configuration state, and the posting

145

object.

146

147

. : Web interfaces

148

. . HTMLFormatter

149

. . htmlformat.py library

150

151

. : Mail interface

152

153

. : Etc...

154

155

.+ The Extension Process

156

157

. - List Admin Options

158

159

The maillist administrator options expose many aspects of a list's

160

behavior to customization via the web. Like the numerous options

161

available to the user of a contemporary web browser, there are a lot

162

of them, so they are segregated categorically in the user interface,

163

in order to ease finding the one being sought. This is a typical

164

substantial new mailman features to entail changes to existing admin

165

options, or additions of new ones. This will be a common facet of all

166

the changes we describe, and

167

168

. : Simple: Adding a new simple configuration variable: reminders_to_admins

169

170

cascading lists - interoperating with both mailman and other MLMs

171

172

. : Also simple, but a bit more widespread: virtual hosting

173

174

Another feature implemented via a few options.

175

176

Implemented with a combination of list-specific and site options,

177

the modular list architecture and structured list admin options

178

mechanism made this easy. [klm: Actually, this was already in

179

mailman when i got to it, but worth expressing how a few config

180

vars enable a whole new dimension.]

181

182

. : Deeper: Adding help for listadmin options

183

184

An elaboration in the structure of the help options, possible

185

because they are structured in the first place. And facilitated

186

by the HTML layout being programmatic.

187

188

. : Deeper still [and pending implementation]: Implement options as a class

189

so they will, eg, defer to defaults values until custom value is set.

190

For instance, to enable a central maintainence of the list of known

191

spammers, forbidden_posters, such that changes to the central

192

copy propagate out...

193

194

. - New procedures:

195

196

. : some effective spam protection

197

198

Many flavors, including the particularly effective

199

require_explicit_destination, worth detailing.

200

201

. . Pending issues: Posting filters should be implemented via hook functions.

202

203

. : integration with external mechanisms (psa membership automation)

204

205

Another kind of extension - use of mailman as an adjunct library

206

from another application.

207

208

By which registration for membership in the PSA invokes the

209

procedures for subscribing members to the PSA mailing lists. This

210

sort of functionality is too general to be able to anticipate it

211

with hooks - MLM programmability was [klm: once i do the PSA

212

stuff:-] crucial here.

213

214

. : list-owners - managing the local community [pending implementation]

215

216

Some restructuring.

217

218

A community inherent in the operation of an MLM site, it makes

219

sense to use the MLM to communicate with the managers of the

220

lists. Likewise, given an extensible MLM, it makes sense to build

221

this into the system, an implicit list. As partitions like

222

virtual domains and cascading maillists evolve, it may make sense

223

to differentiate subsets of these implicit lists.

224

225

. - Whole new subsystems - mail/news, direct smtp, etc.

226

227

. : Power of encapsulation!

228

229

. : Power of the Python library!

230

231

.+ What kinds of elaborations might we want to deal with?

232

233

. - "Subchannels" - more elaborate maillist interrelationships

234

235

. : Collaborative filtering

236

237

Whereby members subscribe to threads based on valuation (e.g.,

238

topical or merit classification) by other members

239

240

. : More elaborate cascading

241

242

. . by category

243

244

. - Shared identification

245

246

. : Whereby an account id and delivery preferences are shared across

247

lists on a site, and even across mailman sites

248

249

.+ Conclusion

250

251

[klm: for the conclusion: In effect, the need for the continuing

252

evolution of an MLM suggests a model with at least parts of the

253

production system - the parts that need to be fluid - perpetually in

254

the prototype stage of evolution. *Mature* prototypes - well

255

exercised and integrated - but amenable to continuing, and possibly

256

deep, change. Prototyping and rapid development are among Python's

257

clear strengths, and invaluable in this regard.]

258

259

.+ Local emacs vars.

260

. - (`outline-layout' is for allout.el outline-mode)

261

. - Local variables:

262

. - outline-layout: (-1 : 0)

263

. - End:

~sambuddhabasu1/mailman/fix_mailman_run_error