Transcript

Olle Johansson: [0:01] For those who didn’t listen to my last talk, I’m still Olle Johansson, and I’m here and now and available for you guys, but not for the rest of the world, presence-wise. [0:11] Last talk was about presence, and gave you a large overview and, really, no answers, no clues, but making sure that you were more confused than before, and maybe a little bit inspired.

[0:25] Now we’re going to look more about buzzwords – unified communication -see more from a business perspective, maybe get some time to look into hints and see where we’re going with this stuff, and try to give my point of view of unified communication.

[0:44] There’s no specification of unified communication. It’s a marketing buzzword floating around: “Everyone is looking for unified communication.” And I think the first unified-communication trade show in Sweden was 15 years ago. So it’s something coming back from the dead, strangely enough.

[1:07] So what is this stuff? Well, it started with telephony. Now we’ve got email and some kind of rude, very crude presence systems for PBXes. I could say that I’m in office, I’m in a meeting, I’m home with sick kids and stuff. But the interesting thing is really to ask where this is going. What’s going to happen with all these technologies in the future?

[1:39] And I wanted to start with showing you a small video, but I have a problem with the audio feed here. This is from the movie “Iron Man.” I don’t know how many of you have seen it.

Man: [1:54] Day 11, test 37, configuration 2.0. For lack of a better option, dummy is still on fire safety. If you gas me again and I’m not on fire, I’m donating you to a city college. [2:04] Nice and easy. Seriously, I’m just going to start off with one-percent thrust capacity. In three, two, one…

[2:14] [loud thrusting sound]

[2:14] [mechanical sound]

Man: [2:26] OK. [2:26] [whirring sound]

Man: [2:28] Please don’t follow me around with it either, because I feel like I’m going to catch on fire spontaneously. Just stand down. If something happens, then come in.

Olle: [2:35] OK. So, if I showed you this movie frame by frame, what would you see? [2:49] He’s talking with a machine. He’s communicating with a machine, like it was a human being. He’s sending voice commands. There’s actually an Asterisk sticker on this machine. Right? Speech recognition, speech synthesis. Asterisk, not a PBX. Asterisk is part of a communication platform, running.

[3:21] There’s a lot of interesting stuff. There was another video sequence I wanted to show you but couldn’t really copy in a good way, from “Star Trek.” One guy sitting on the ship, playing a game with a lady in a different part of the galaxy. The whole wall is a video screen. He’s eating some food. They’re playing games. They’re communicating.

[3:55] And it’s a magic scene for everyone who works with communication, because it’s presence. They communicate as if they were sitting in the same room. They’re not forcing communication. They’re very relaxed. They’re playing games. They’re discussing. I think he’s going out to get a cup of coffee and coming back.

[4:19] So why is this so interesting? Well, there’s no charge per minute. They can actually be together over video communication. And it’s not that far away.

[4:33] So, movies like this, apart from being amusing, shows us where to go. And, again, I’ve been preaching for over 20 years now what’s important.

[4:50] So I’m going to preach for you for a long time, but let’s start with a traditional slide, about me.

[4:59] [speaks in Swedish]

Olle: [5:31] OK. One guy left. [5:35] [laughter]

Olle: [5:36] So, we had serious interoperability issues. Problem is, my German is as bad as your Swedish. That counts you Danish guys over there as well. [5:49] [laughter]

Olle: [5:50] So we have to find a way to communicate that you can understand it, I can understand. We have to reach interoperability. And interoperability rocks. That’s cool stuff. That’s what creates application systems, unified communication platforms, that will take us to the future. [6:15] But let’s start by looking about where we are, try to get a status report. What’s going on?

[6:23] First, sad story. IP telephony is nothing new. The SIP protocol is over 10 years old, from the RFC 3261. And we’re still confused about how to parse that document. IP telephony has been around for ages.

[6:43] It exists in cell phones, phones, everywhere. Takes 10 years for something to reach wide implementation, and we’re there now. Ordinary carriers are looking at IP telephony trunks. IP telephony is everywhere. Don’t come to me and say, “It’s something new. It’s sexy. It’s cool.” We’ve been doing that for many, many years.

[7:06] And, sorry, open-source VoIP? Come on. Nothing new there, either. This year, Asterisk will turn 10. Come to Phoenix and celebrate with us at AstriCon this fall. It’s going to be a big, Asterisk, 10-years party.

Many years of guerrilla work. When we started this off, and I joined the project after a few years, we really felt like a guerrilla: [7:25] “Oh, we’re going to kill AT&T. We’re going to kill Telia. We’re going to kill all the carriers! We’re going to take over and run this over Internet.” Guess what? The carriers took it over, and they’re running it. And we’re the guerrilla. Hello!

[7:46] So, the guerrilla’s now growing up, working with the carriers. And they’re using open-source VoIP. And we have many applications. And you saw the features from Mark. We have a huge part of the market.

[8:07] And the really sad part for us working with Asterisk is that, while we were heroes a few years ago – Asterisk was the coolest stuff on earth! Hey! Asterisk is still very cool, please? Yes? It’s cool? Thanks.

[8:29] But Asterisk opened up a new way. People are building incredible things, powered by Asterisk. They’re building PBXs with web front ends integrating into presence and stuff, messaging, Google Maps, sales systems, CRM systems, whatever. But it’s not Asterisk. It’s accepted technology, we’re there, we’re mature, we own a large part of the market. Boring.

[9:05] So, VoIP is nothing new, Asterisk is nothing new, so why are we here at Amoocon? Well, we’re the winners. We have to meet and say, Hey, we’re the best, we’re the winners. And look at what next.

[9:24] And also to realize that for our customers it’s an important message that this is nothing new, this is well established, we’re market leaders. Technology, TC protocol and all the rest of the stuff; open source VoIP, nothing dangerous there; a large business infrastructure, companies that actually work like traditional companies exchanging bills and papers and money to do good services.

[9:56] But it’s not a question about VoIP protocols anymore. SIP is the protocol. And now everyone talks about Unified communication, even me. How cool is that?

[10:12] Let’s talk about open. When you look at a platform, open network protocol I think is the most important. Open source comes in as a secondary piece for me, but in some cases we go offer closed source, we offer proprietary switches. It might be more efficient, might be cheaper, I don’t know. That’s another discussion.

[10:39] But the most important for me is that when you build a long-term infrastructure you need to secure that infrastructure by using open standards so everything can communicate with everything.

[10:52] You should be able to install a VoIP system and later make decisions about which phones you want to use. As long as they follow the open standard you should be able to take, OK, Cisco is here because they’re great now, and Snoms next week, and Soyo for here and something else here.

[11:11] Open standards protect your investment. Open standards put you and your customer in control, not the vendor. And that’s extremely important, especially in these times. And again, interoperability.

[11:35] A lot of people claim they’re using open standards, but they tweaked it a bit, or it doesn’t work. They haven’t read the RFCs, they haven’t gone to SIP it, they haven’t tested it, they haven’t made sure that things really work together.

[11:49] And as a customer you need to require interoperability, and proven interoperability. Make sure that this phone does work with other servers than servers from the same manufacturer. I think that’s critical.

[12:07] I mean, imagine email software today that didn’t work with a mail server and didn’t connect to the Internet. “Oh, you know I have a Gateway server here in my network and you have to pay me a euro per mail, incoming and outgoing.” Would anyone even consider buying that today? Yeah, spam is what would drive my revenue.

[12:35] So, open standards and interoperability, priority one. Then we have open source. That’s an interesting area because some people are not really used to the business model of open source and how it works, the ecosystem behind it powering.

[12:59] I had a customer ask me the other day, we were discussing a new project and he said, “Well, oh, you’re this huge global one person company. What happens if you disappear?” I was like, "Well, there’s probably 100 companies in Sweden working with the same stuff and they all have the source code, and they compete with the same information. I might have more experience, next year they might have.

[13:29] So you have an open market. If you buy something from Cisco, you have Cisco Sweden as your partner. That’s your only source of services."

[13:41] “Oh, interesting.”

[13:45] And coming from Sweden, wow, we have a lot of adoption we need to do. We have a small language as seen from worldwide. We have a lot of other things. And as a software distributor and system integrator I can fix that or I can ask someone else to fix that. I don’t have to go to Digium every time.

[14:08] A lot of people don’t understand that. They say, “Oh, we have to speak with Digium.” Why? If you need help with their expertise, the cards and the core software and stuff that we can’t find anyone else, fine. But otherwise we can ask several people including Digium for help. And that’s open source, a very capitalistic marketplace.

[14:28] So there’s a lot of cool stuff with open source and I’m sure you know all of it. I’m preaching to the choir here. And when communicating with customers you need to turn that into business terminology. I like to say that empowers your business, you get control.

[14:49] You can choose service providers, you can choose support providers, and you can feel free. If you want to adopt a software, it’s your business.

[14:57] If you want to get some extra benefits of system integration it’s very, very simple compared with buying, call manager from San Francisco, hey, try to adopt that and tweak some things in that, ask them for the source. You are in control of your business in a changing marketplace.

[15:21] So, what should happen now then? If we agree upon open standards, open source, that’s the way to the future. It puts you in control. It makes business sense. It’s safe now. Remember Morris [inaudible 15:38]. It’s no big risk. It’s must more risk buying a Nortel system than buying Asterisk today if you see it from a commercial perspective.

[15:47] The third thing I would like to tell you is stop building PSTN over IP. It’s cheaper. If you want that, it’s cheaper to go buy a secondhand Siemens PBX or Alcatel or whatever and install that, OK? Stop doing this.

[16:07] When customers start looking into going VoIP tell them that we’re actually building now a real-time infrastructure. You need to have an addressing plan, you need to have a platform that can not only handle the old telephone applications, but future applications.

[16:28] The only thing you can say for sure is that you don’t know what this platform will be used for in the future. Remember how broadband service providers in the mid-90s, they calculated their network and said, OK, I’m selling 10 megabit to each apartment, but that’s fine.

[16:46] I only have three megabit to the Internet from the building but I have a caching web proxy so they won’t discover. So they’d assign their network for one application, the web.

[17:02] They were crying when file sharing and peer-to-peer video and stuff like that came up because they couldn’t overbook, their whole platform built for the web just crashed.

[17:16] So, stop building PSTN over IP. The PSTN guys are all busy doing that by themselves. Every guy in the PSN world that adapt to VoIP, they still think SS7 over IP, or PSN over IP.

[17:34] And remember Presence. Even if the customer or you don’t want it today, it will come. And you don’t want to have a set of separate islands. You want to connect these, make them work together and fit the business needs, and that’s going to be a problem.

[17:57] You can decide what the company internally is using, but you still want to interface with the world of Presence systems out there, somehow. But looking at this picture, I only have one comment; “grrrr”.

[18:18] How did this happen? Well innovation. Everyone wanted to build their own system. Can we fix this? Don’t know. But, let’s look at what we’re about to create. We want to have one real time platform in enterprise.

[18:41] We want to have Presence, VoIP and all the things running there. I think all of you have seen these pictures but, for me, I keep judging SIP and Jabber. And SIP for Presence and instant messaging.

[19:03] I’m sorry Yoakim but there is no huge movement there. I’m real happy that Yoakim over there from Securex, gave me SOIPER, that actually is a simple client I can use and test stuff with. Because I only had one client before that. And I have a lot of applications in my computer. But, this is something we need to work on.

[19:27] As I said earlier, social networks might replace the whole thing. It might be Facebook or Skype or something else that takes over, and SIP and Jabber and MSN will be gone. But, one thing is for sure, is that when people built web applications, they designed it for the PC. Well the PC is no longer the major Internet client.

[19:57] We have iphones. We have all kinds of devices attached, including my sweet little rabbit back home connected to WiFi, saying funny things and repeating clocks and other things. There are so many devices out there that are not PCs.

[20:12] When you look at the real time system, the phone will not be a majority of the clients. When I work at my desktop, my desktop is the control center. The phone is just an order device. When I’m out, it might be my cellphone, it might be some system in my car, it might be something else. But don’t design this for phones with handsets.

[20:45] I honestly also think that, I haven’t seen Jay Phillips here so I can safely say this, text based communication will control here. Web pages is not the answer. Remember the video I played. The human speech interface will get more and more important in all of this; Speech recognition that kind of system.

[21:19] It might be science fiction for some of you today, and it might be for another five to eight years or something, but, it’s been in Star Trek since the beginning. “Computer on screen.” Speech control of computers.

[21:36] My son was screaming every night for two weeks in his room trying to get speech recognition working in his Windows Vista system. And the whole family was so tired with that stuff so we actually removed his microphone for a while.

[21:55] But, as a kid, he feels that’s the natural way to control it. While gaming on one computer, talking with the other one, while controlling web surfing and other things.

[22:07] I might be told to accept that. But, I think, when building systems look at it. So, Stefan asked me to combine a few talks into each other. The big missionary, evangelist, blah, blah, blah stuff with some practical information, down to earth.

[22:30] So I added a few slides about one of the projects I’ve been working with on the Fall, during the Fall also down in Portugal. Why Portugal? Well, if you go to Sweden during the winter you get the answer. It’s dark, dark, dark, cold, cold, cold and extremely expensive wine and food. Portugal the opposite, right? Right. Nice place to be.

[23:01] Let’s look at how to build this stuff. Every time you hear stories like this, evangelist like me speaking, you have to understand where they are coming from. I’m coming from the data comp side. I’m a net head. Other guys might tell completely different things because they have a traditional “how to build proper telephony”. They’ve been doing that for many, many years. They’re the bell heads.

[23:28] So, for me, when I build systems, I don’t want to think traditional playbacks. I don’t want to think PSN, and I want to listen to users and see how they’re using the system and help them build applications. I think that’s important. And I keep drawing analogies with email and instant messaging when I talk about Asterisk and how to implement Asterisk.

[23:59] One of the projects I’ve been working with that is just about to get a user base, we worked all Fall for this, was a real time communication platform, a unified communication platform for all students in Portugal.

[24:18] What they’re doing at FCCN, which operates the National Academic Network, is building an authentication federation based on Shiblif[sp] Simul 2.0 authentication. Every student in that system, every university that connects to the federation, will get accounts immediately in real time communication platform.

[24:43] As soon as they have an account at any university in Portugal connected to the federation, they can go and download client, which is Bria from Counterpass, and they will instantly have SIP with video/audio, and they will have Jabber, Presence and instant messaging.

[25:06] So this was designed and stored for 50, 000 users. Our requirement was that we should be able to scale, by adding more computers and stuff, to 250, 000 users. A very interesting for me, because I normally spend my time with carriers and that kind of stuff, is that we had no PSN connectivity.

[25:28] I’ve been talking about this real time, unified communication platform. Stop thinking PSN. And I got a project like that. There’s also legal implication, because, if we’re not connected to the PSN, all the legal issues disappear. We’re operating just an Internet application.

[25:51] So we can do different things, build differently. We don’t have to provide emergency phone calls. We don’t have to provide any espionage services for the state or things like that.

[26:05] So this was really cool. We started stacking up servers. I think this picture…we’ll show all of them. Eleven servers for different things. So, in the front we have Kamailio SIP Proxys that handles all the communications.

[26:26] Of course, we have DNS and Stun service as well. We have the SIP registrars’ location service that handles all the SIP location issues and forking and that kind of stuff.

[26:39] We have Asterisk running on voice application services and, of course, they’re limited in regards to how many calls they can run per server. But if we need more calls we just add them.

[26:51] Calls between users go between users. There’s no media gateways here. If they have NAT, we have RTP Proxys, but that’s just… remember this is for university. They have iPod resist. They have all the iPod resist [?] in the world, so in most places there’s no NAT, but if they connect from hotels or from homes, we need to have RTP Proxys for that.

[27:15] We have three Jabber servers running ejabberd from ProcessOne, an Open Source Jabber server that’s scales like crazy. And this was a small installation for ProcessOne, which is kind of interesting with 50, 000 users. Then we have the rest of the servers. We have MySQL servers, we have web servers, and all of that running on the leftovers in the bottom.

[27:42] Basically the user logs in to a portal, authenticates in a federation. That means they get redirected to their local university authentication site, comes back, and downloads the client if they haven’t installed that, or initiates the client from the web portal, because it’s important that all communications start with a Shibboleth log-in to get the proper authentication.

[28:07] We can’t do that in the SIP and Jabber protocols today. We have to initiate all communication from the portal. In the back end we’re operating a lot of PKI stuff, so we shoot and certificate some keys to the phone, and the phone uses that to get to the provisioning server and get all that cap data, and blah, blah, blah, blah.

[28:28] But it works very, very nicely, and the building blocks weren’t Microsoft, or Cisco or anything like that. It’s all Open Source. So, of course, we have OpenSER, Kamailio, ejabberd, and Asterisk as the main applications all powered by Centos.

[28:51] We have Nagios, Cacti, Rsyslog in the back end. The portal runs PHP, Apache, MySQL – everything here connects to MySequel. MySequel used to be Swedish, moved to California, and now I don’t know what’s going to happen.

[29:06] And, of course, OpenSSL is everywhere here, as well, and probably a long list of other things. But this wasn’t in the bid requirements. Other companies offered proprietary solutions to FCCN.

[29:22] But we won on business terms, but I think both sides liked that we’re using OpenSource. You have seen a lot of contributions to Asterisk, lately, from this project and another project we were running in Portugal at the same time. I can mention that later.

Participant: [29:42] [off mike comment]

Olle: [29:44] The soft phone, no I didn’t mention that. We were really looking for Open Source soft phones, but there was no one that was really ready for production use on Windows, Mac at the time.

Participant: [29:59] [off-mike comment]

Olle: [29:59] What?

Participant: [30:00] [off-mike comment]

Olle: [30:01] We didn’t find you. Maybe we can find you now and add you, so we don’t have to pay licenses. That’s a sore issue. I can say without affecting business relations that that was the most complicated part of the whole deal to get that team into the picture, because they were thinking different ways of doing things. [30:32] This is another way of looking at it all. Yes, Matt.

Matt: [30:35] [off-mike comment]

Olle: [30:39] … You’re saying? Sorry, I have a problem with language. It was a requirement to use SIP and Jabber in the solution.

Matt: [30:50] [off-mike comment]

Olle: [30:52] Sorry?

Matt: [30:53] Who did you find?

Olle: [30:54] Bria from CounterPath. OK? In a special version that includes all of this Shibboleth authentication. [31:04] When building this, there were large requirements to scale, so we were… it was very important for us to see every service as a separate service that can move between servers as we scale up.

[31:22] We address every component with a virtual DNS name instead of using the host name. So if we move SIP proxys, or add SIP proxys, we can add that in DNS. So DNS is extremely important here.

[31:36] We worked with Subversion the same way, so every component has their own directory with installation scripts, and everything runs automatically with M4 and Subversion, and a lot of scripting, and Diet Coke, I guess.

[31:53] For redundancy in scalability, well, there are plenty of choices and opinions out there, but we took a simple route, much the same as we used in many carrier installations. We used VRRP for the order between servers.

[32:12] This is, I would say, the Corsi proxys, the web, the provisioning server, and MySQL are all running behind VRRP. This means that if one server dies, the service moves to another box within seconds.

[32:31] Of course, we have DNS Service records as well, and Jabber SRV records, and so we have fill over and low balancing through that.

[32:43] For the databases – I’m not a database expert – we looked at many different options. We ended up in this project initially using MySQL proxy and two MySQL servers with bi-directional mouse-to-mouse replication.

[32:59] It turned out that this worked well with all applications except ejabberd, because if you run that way, you have some applications, first in a query to check if a record exists, and if it doesn’t exist, they do an insert.

[33:13] With MySQL proxy, the query can be in this server, and the insert here, but that record was inserted here a second before that, but not replicated yet. So the application should really have done an insert, and when the insert failed, do a query to get the data.

[33:32] It seems like Asterisk and Kamailio behaves much better. So, because of ejabberd, we decided to remove MySQL proxy and mover to VRRP instead and have scripts that monitor MySQL status and health and other things.

[33:51] Voicemail, IVR, well, OpenSER takes care of that, so we don’t need VRRP. If something fails SIP-wise one Asterisk server is open-zeroed automatically and moves to the next one and disables that one. That works with very nicely with the SIP proxy.

[34:13] The rule of thumb when you build systems like this, is that, messaging-only servers like SIP proxys scale extremely well. I mean, four servers per SIP here means that we can handle a massive amount of traffic. It might be different if you had Presence or dialog states. We have a lot of states in the SIP proxys, but as long as you don’t have call or session states, four servers is much more than we want.

[34:45] But the servers that handle media are the IP Proxys and Asterisk, well, there we have a problem. Mike [name] who whispers, and I can probably discuss for hours, how much media one server can handle. But, we are reaching a point in all applications now – Asterisk is a free switch – and other applications where the media flow kills the CPU, and the Linux kernel.

[35:14] So, it is not Asterisk or free switch or applications anymore running on a quad-core. The Linux kernel crashes when Asterisk is running at 26%, if you only have VoIP calls and only media.

[35:31] So, there is a lot of stuff to handle there, and you need to make sure that you have scalability by adding more servers.

[35:42] Conferences, I mean voicemail, we can handle in any server. It doesn’t matter; it is not a state machine. We have voicemail over email, but conference is a problem, because all the people in the conference probably want to be in the same conference on the same server.

[35:58] So, we basically did some hacks and made sure that when we start a new conference, we add a record in the MySQL database so that OpenCA can locate the conference when someone calls.

[36:14] And, when the conference dies, we remove that record and everything is fine. Then we can look at max load. Asterisk has settings for max load and other things. So, Asterisk can deny a new conference call as well, if it is overloaded.

[36:33] We are pretty fine there, we think. We will see what happens when the system really gets loaded.

[36:40] So, why XMPP and SIP are coming back? This is part of another presentation I have set up but XMPP, felt by the customer, the university network, has much more traction for instant messaging presence than simple. And, SIP unfortunately is only used for calls. I think it is a big mess out there.

[37:04] In addition to this project, there was another larger project that I haven’t got any slides on yet, but basically we have taken a majority of the universities, and all the institutions and ripped out all of the PBX’s from the PS down.

[37:23] In front of every PBX in every institution, we have an Asterisk, with either Digium BRI cards, or St. Roma BRI cards, connecting to PBX. Then, we are forwarding the calls over SIP to Chameleon SIP proxy and we have a session board controller.

[37:43] So, we are moving all the trunks to SIP trunks, and that way this saved 5% only by using Enum between the universities, 5% cost. And now, they think they can save between 30 and 40% by moving to SIP trunks. The next phase of the project will be to actually install SIP phones. So, we installed in every university an IP PBX that can handle all the phones.

[38:11] So in total, I think we installed 300 servers. I would say that 40 of them were session board controllers, 40 billing servers because the university had in it internal billing, but the rest of the servers were all running one Asterisk and one Chameleon.

[38:32] So, let’s look at the vision then, where are we going? Components of the open unified platform, open network protocol, open source and even more important, new APIs for your new applications.

[38:52] Remember, we need to build new applications. The only thing I know about this university cool stuff is that we are not cooler than Amazon and all that stuff, but applications the students will build on this platform. Those will be really really cool!

[39:13] So, after the summer we hopefully are going to get funding to start a project where we are going to promote student’s innovation on the platform. So build new applications, look at the new platform that is coming your way with all the cool new APIs up there to build it from.

[39:37] So, let’s finish up with some executive summary and repeat all the stuff so you won’t forget what I said. I am saying the same thing over and over again, so when you are asleep tonight you will dream about this stuff.

[39:50] Conclusion number one, open standards protect your investment, right. Number two, open unified communication gives you freedom to control your communication platform, even more important.

[40:07] And this creates a very, very competitive business environment which is good for the customer. There are many, many companies creating Asterisk call ware. There are many, many companies and projects out there. There many companies supporting all these different applications – service providers, system integrators, support, training, whatever.

[40:31] We don’t own our customers. Open source will take you there. That is really important, and it is really important to understand that we have no clue where this is going. It might be 3D holographic applications. It might be talking to my robots, like Ironman. It might be Star Trek; “On screen please. Oh, hello, Kevin Fleming. What is up today?”

[41:02] We are building a platform for new applications. Don’t limit it to one application. It is not PSTN over IP. This is where PSTN over IP is going, it is dead. This is all about connecting users in real time. Thank you.

[41:21] [applause]

Olle: [41:27] Any questions, by the way? Preaching time is over.

Participant: [41:37] What is Cacti?

Olle: [41:38] Cacti is like MRTG, a statistics analyzer. It shows us all the diagrams of traffic and hosts and everything.

Participant: [41:47] [off-mike comment] .

Olle: [41:56] OK. I think my company is smaller than yours.

Participant: [42:00] [off-mike question] ?

Olle: [42:25] Well, that’s easy, Mark Spencer. Who am I going to sue if this fails with open source software? Sue Mark Spencer. [42:33] Well, I think that an old way of seeing it, but it might be that all of us that work in small companies, in order to fight the bigger companies and the security they give, we have to build some kind of business infrastructure between us. So, we can say a chain of stores that we are actually much bigger than my company.

[42:59] We are conglomerate of companies that can take these big deals because otherwise the big deals would go somewhere else, and that’s not good for either one of us. But, that is a business issue I really think we ought to discuss.

[43:13] The Portuguese deal we won, there was a Portuguese company fronting us that did all the installation and integration. They had the power to go to every university and install all the boxes.

[43:26] We were in the Aurora project. It was me in Stockholm, Morten in Oslo, Daniel and Ramon in Romania, Phillipe in Paris, Caleb Jaffa from Utah sitting in Falland, Sweden up in the snow and cold so he had a little time to focus and develop. So we were a large team – and of course Pedro and Rubin in Portugal as well.

[43:54] So we were a large team all over Europe of companies working together but acting as one towards the customer. That gave us the opportunity to win this pretty large bid. I think we have to get use to working like that. It makes business sense and we all can get sued by the customer. Any other questions? Matt.

Matt: [44:19] [mumble]

Olle: [44:29] What is in…? Handling media in the servers.

Matt: [44:32] [inaudible 44:32]

Olle: [44:40] [laughs] Well, the media servers that handle stuff basically Asterisk is voicemail and IVR is into and conferencing. [inaudible 44:49] is just media streams, video, audio, whatever that needs Novatel. [44:55] But we did extensive testing with equipment from Novatel and a large team of crazy engineers from Kernel hackers to Asterisk hackers and Linux specialists. And we were able to run on a dual core system 1600 channels dual and 800 calls.

[45:17] After that… And we trimmed a lot of the network drivers and stuff in order to get there. At start we were at 500, 600 calls. After that the Linux Kernel, the RQ balance and all of that just gave up and the kernel got all the interrupts on one CPU and everything just died, all the voice quality.

[45:38] But when we took down the volume, Asterisk survived and it could take another hit and up and down again. So Asterisk has become much more stable. I was very impressed over that.

Participant: [45:48] [mumble]

Olle: [45:50] One-four. I don’t remember. Sorry. [45:56] It was One-four with a set of Aveena patches. But basically the test was with a SVN I believe. But patches does involve in the core threading so performance wise it was a plain Asterisks. Some people have told me on the mailing list that after that if you are on multiple Asterisk servers that you can take much more than 1600 channels.

[46:20] We are a bit curious about that because we have been speaking with many propriety software vendors building firewalls and other stuff and they have kind of the same data. But we would continue testing.

[46:32] We want to reach at least a 1000 channels, not the 10, 000 channel that John Shultz had asked me about. But there is a problem. At Asterisk DevCom in Georgia, I believe, Steven Yueller from Sun confirmed that he saw the same on his massive 64 processor Sun Systems.

[46:54] And he went to the Kernel developers, the stack guys and they said, “Get out of here. We’ve been working with optimizing for the web for 10 years and we haven’t bothered with you to pee and know you’re coming with this lousy shit. Go away!” So You too pee performers into [inaudible 47:12] stacks really needs to be looked into.

Participant: [47:14] [off-mike speech]

Olle: [47:23] Because you never answer your cell phone! OK. So how did you do it? You do it as a Kernel module? Right.

Participant: [47:32] [off-mike comment]

Olle: [47:35] Yeah. BSD can handle a much more load.

Participant: [47:39] [muffled]

Olle: [47:41] What?

Participant: [47:42] [muffled]

Olle: [47:43] Yes. There is actually a new stack system coming up in FreeBSD, I believe, 8, that looks extremely impressive. So we can have virtual stacks and multiple stacks and stuff. [47:59] So I think we can do a lot of that, but what I would like to do at some point is have something I call Astrotech spend one week with people like you, Michael, Leonard, Digium guys, really beating the hell, and test tools that can actually give us load, and beating the hell out of Linux and Asterisk and Freeswitch and all of that and see what we can do.

Participant: [48:23] [off-mike question] crashing the Linux kernel?

Olle: [48:28] Not really crashing, trashing would be a better phrase.

Participant: [48:31] You’re not crashing the machine?

Olle: [48:34] No, no, no. But all our queues moving to one of the cores, and that core gets overloaded, but as I said, when we take down the volume, everything moves back.

Participant: [48:46] [mumble]

Olle: [48:48] 1600 channels of… [48:50] I have, I am very bad, poor at remembering number, but I remember it was 1600 channels of URL [?]. I don’t remember the packet rate.

Participant: [48:59] SIP to SIP?

Olle: [48:59] SIP to SIP.

Participant: [49:01] With medium?

Olle: [49:02] With medium. No trans coding. No listening for DTMF. I mean, a very simple bridge in Asterisk [mumble] .

Participant: [49:12] [mumble]

Olle: [49:15] OK.

Participant: [49:15] [off-mike] We’re up to 500 calls.

Olle: [49:21] Oh, so I’m better than you, but I’m not making it as good as Yule [sp] .

Participant: [49:24] [off-mike speech]

Olle: [49:25] Number two.

Participant: [49:26] But on the other [inaudible 49:34] , or…

Olle: [49:37] That’s what I want to figure out. I want to figure out if there’s something we can do in Asterisk. I get so many messages. I get Stephen Yule, with Solaris. [49:47] I get other guys doing proprietary stuff on Linux systems that handle media, video servers and all kinds of stuff, and they all confirm that this is a problem for them; but, Michael, saying 3000 calls per system of something.

Participant: [50:03] [inaudible 50:03 to 50:19]

Olle: [50:18] Yes, I mean, if you’re in the kernel, you don’t have the context switching issue, you only have the interrupt issue, and I think you can go much higher then. So, sue us, or the people that’s in the kernel is interesting, and that must be the GPL right?

Participant: [50:34] Well, [off-mike comment]

Olle: [50:41] Yes, connection tracking. Yes, I know, but then you’re executing in the kernel and that means that you can take much more because you don’t have the context switching you had in Asterisk between use of space and kernel space in the process; but moving all of Asterisk to kernel space, I’m a bit afraid of that. [51:02] That would mean we can halt the system and still operate the Asterisk, which would be really fun. So?

Participant: [51:07] [mumble]

Olle: [51:15] That wasn’t anything I was involved in. It was convergence. A Telco class system from somewhere.

Participant: [51:28] [mumble]

Olle: [51:29] Proprietary solution, yes.

Participant: [51:31] [mumble]

Olle: [51:35] What the SPC really does is a topic that I would love to discuss with you guys for a long time, because I’m really confused about that market, but one of the things they do is earn lots and lots of money. [51:49] So, I have to let the next speaker up on stage, take over the Madonna tool kit and the proctor. Thank you very much for listening. I’ll be around today and tomorrow before lunch as well, if you have any questions, discussions or issues you want to discuss. Thank you.

[52:04] [applause]