Usando MongoDB como storage de arquivos #1

julho 20th, 2011 by caferrari Leave a reply »

Estou envolvido em um projeto que vai necessitar armazenar muitos arquivos, e precisava de uma solução que fosse tolerante a falhas e escalasse muito bem, tanto verticalmente (Mais disco) quanto horizontalmente (Mais servidores).

Depois de olhar algumas soluções existentes como MogileFS, Cassandra, entre outros, eu gostei bastante do GridFS do MongoDB, pela simplicidade de configuração e principalmente pelo sistema de eleição de um novo servidor mestre caso o ativo venha a falhar em um determinado replicaSet.

Minha configuração inicial dessa estrutura consiste em dois servidores de Storage (Ubuntu Server 11.04 64bits rodando em VMs vmware em discos diferentes) e uma terceira maquina simples e remota que será um backup onde será configurado meu primeiro ReplicaSet. No servidor de aplicação vou ter o MongoDB rodando como arbitro (sem armazenar dados) apenas para determinar qual o servidor é o mestre caso algum venha a falhar.

Instalação:

Considerando que nos servidores foi instalado o ubuntu server, sem nada, vamos a configuração:

primeiro vamos adicionar a linha na configuração do APT para podermos baixar o pacote oficial do MongoDB, mantido pela 10-gen, crie o arquivo: /etc/apt/sources.list.d/mongo.list e adicione a linha:

deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen

depois de salvar o arquivo, adicione a chave do repositorio:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

e finalmente instale o mongodb:

sudo aptitude update
sudo aptitude install mongodb-10gen

repita este processo em todas as maquinas de seu replicaSet.

Configuração do MongoDB

Primeiro vamos criar a pasta de armazenamento dos dados e definir as permissões:

sudo mkdir /dados
sudo chown mongodb:mongodb /dados/
sudo chmod 700 mongodb -Rf

Em todos os servidores vamos editar a configuração do MongoDB em /etc/mongodb.conf, aqui ficou assim:

dbpath=/dados/
logpath=/var/log/mongodb/mongodb.log
logappend=true
replSet = arquivo1
rest=true

* o parâmetro rest, faz com o que o servidor crie uma página para monitoramento dos status.

Depois de configurar todos os membros do replicaSet, reinicie o processo em todas as maquinas:

sudo service mongodb restart

Agora temos os servidores prontos para configurar a parte mais simples, o replicaset

Configuração do ReplicaSet

Para facilitar, configure nomes para seus membros do replicaSet no seu servidor de DNS interno ou, edite o arquivo /etc/hosts e adicione as maquinas que serão membro do replicaSet:

192.168.0.100      arquivo1a 
192.168.0.101      arquivo1b
192.168.0.102      arquivo1bkp

* Ajuste os IPs de acordo com sua rede.

Agora, em um servidor qualquer, acesse o console de administração do Mongo:

mongo admin

e crie a configuração do replicaSet:

conf = { _id : 'arquivo1',                    
... members : [
... {_id: 0, host: 'arquivo1a'}, 
... {_id: 1, host: 'arquivo1b'},
... {_id: 2, host: 'arquivo1bkp', priority: 0}
... ]}

* Note que o servidor de backup tem prioridade zero, isto faz com que este servidor nunca seja eleito como servidor Mestre.
confira se está tudo ok:

{
	"_id" : "arquivo1",
	"members" : [
		{
			"_id" : 0,
			"host" : "arquivo1a"
		},
		{
			"_id" : 1,
			"host" : "arquivo1b"
		},
		{
			"_id" : 2,
			"host" : "arquivo1bkp",
			"priority" : 0
		}
	]
}

se estiver ok, ainda no console do Mongo, inicialize o replicaset:

rs.initiate(conf)

em 10 segundos o seu replicaSet deverá estar rodando e pronto para receber dados, veja como está pelo navegador, acessando:

http://192.168.0.100:28017/_replSet

Você verá uma tela com o status de todos os membros do replicaSet:

Bom, é isso, no próximo post vou explicar como configurar o php e efetivamente armazenar arquivos no GridFS, Abraços

Advertisement

278 comments

  1. sex disse:

    It’s an awesome post designed for all the online visitors; they will get benefit
    from it I am sure.

  2. More often than not, users at the online resort booking sites don’t know concerning
    the coupons being supplied, their validity,
    where to get them etc. It takes numerous time and power to find a coupon for the specific
    portal, which is valid and meets the factors of their staying needs.

    At we solve the issue for you, when you come to our website and visit the ‘Hotel Coupons ‘ page underneath the
    ‘Travel Coupons’ tab.

  3. Hi, yes this article is genuinely pleasant and I have learned lot of
    things from it regarding blogging. thanks.

  4. Manuel disse:

    Before you use night cream, make sure you had cleaned your face at the very least two times throughout the day
    and also your face must be clean of make-up during application. showers|showers I
    discover bathing at night and also in the morning is the very best method to go, in the evening i have the deluxe of washing or lengthy shower while in the morning i take
    a fast shower.

  5. Please call 1 (800) 819-5053 for assistance.

  6. But what units it aside from different websites is the fact that you possibly can even specify the locality inside
    the metropolis that you just plan to visit.

  7. It’s manner higher than the AAA charges on the resorts’ own websites and even gathering the lodge chain’s
    loyalty program factors (HHonors, Marriott Rewards, and many others.).

  8. 栃木県のネックレス買取栃木県のネックレス買取

  9. This is the very first thing rental automotive brokers are educated to do.

  10. It really is not a spot for a girl to be alone.

  11. Excellent website. A lot of helpful information here.

    I’m sending it to some pals ans additionally sharing in delicious.
    And naturally, thank you in your effort!

  12. Hi there Dear, are you truly visiting this website on a regular basis, if so after
    that you will absolutely obtain fastidious knowledge.

  13. http://sciences.uodiyala.edu.iq

    uod law,uodiyala law, بواسطة كادر كلية القانون معاً من اجل الوصول للمركز الاول للجامعة عمل اضافي لكلية العلوم

  14. As a result of these, company are in a position to respect scrumptious
    meals without having to exit of the lodge.

  15. I read this piece of writing fully regarding
    the resemblance of hottest and earlier technologies, it’s awesome article.

  16. http://colarts.uodiyala.edu.iq

    اوس الخزرجي,المهندس اوس الخزرجي,Aws Alkhazraji,Colarts Diyala

  17. http://law.uodiyala.edu.iq

    المهندس الاول في عمل السيو في
    جامعة ديالى “اوس الخزرجي”.hero engineer Aws ^_^ Alkhazraji

  18. http://engineering.uodiyala.edu.iq

    المهندس اوس الخزرجي1,الاول في مجال السيو ومحركات البحث1,حاصل
    على درعي جامعة ديالى لتصنيف الويبوكاتركس 1اوس الخزرجي,Aws1 Alkhazraji

  19. Planning a birthday party at a lavishing and superb resort?

  20. Felica disse:

    Laarge numbers of manufacturers have enhtered into the marketplace for offering the absolute best, distinctive inds of trackers tto the customers.

    Also visit myy site :: Felica

  21. Ramonita disse:

    Please let me know if you’re looking for a writer for your blog.

    You have some really great posts and I believe I would be
    a good asset. If you ever want to take some of the load off, I’d absolutely love to write some material
    for your blog in exchange for a link back to mine.
    Please blast me an e-mail if interested. Many thanks!

  22. 福島県のマッサージの使う押し迫るとは。お役立ち敷き地です。福島県のマッサージの細かくはこちら。手記を完投する。

  23. @musicallyapp hasn’t Tweeted yet. Dear customers Musically.

  24. Good pоst.I leearn something totally new and challenging on blogs I stսmbleupon eveгyday.

    It’s always helpful to read through content ffrom
    other authors and uѕe something from other websites.

    Here iis my blog post drain gratе manufacturers; tradingspot.ca,

  25. Hmm it appears like your site ate my first comment (it was extremely long) so I
    guess I’ll just sum it up what I wrote and say, I’m
    thoroughly enjoying your blog. I too am an aspiring blog writer but I’m still new
    to everything. Do you have any tips for first-time blog writers?
    I’d really appreciate it.

  26. Forrest disse:

    Hello! Marvelous comment! I appreciate the way in which referred to Usando MongoDB
    como storage de arquivos #1 > Carlos André Ferrari.
    One more superior article away from good blogger. Get the information most people blog site!
    It usually is intriguing to read through you works.
    If only My partner and i these superior writing ability.
    I’m a undergraduate but i will need create articles tons of written documents always
    .
    A neat you’ll find be internet websites like this one Forrest which in turn gaining product reviews
    with regards to various article having providers

Deixe uma resposta